User documentation¶

Scheduling and data transfer protocols¶

The Perfect Streamer program is designed to transmit MPEG-TS streams over the public Internet with packet loss and delays based on UDP.

For each MPEG-TS stream (Stream), a transmitter server (Sender) and one or several receivers (Receiver) are configured, hereinafter the bundle is designated Peer.

Configuring the transmitter and receiver comes down to entering a list of streams (Stream) and settings for each Stream of the list input and output. Multiple inputs in the list provide redundancy for sources. Multiple outputs in the list allow streams to be sent to different destinations at once.

For a transmitter, input is the sources of MPEG-TS streams, output is the transmission of streams to receivers. For receivers input - receiving streams from transmitters. Four Peer protocols are available to transfer streams between transmitter and receiver:

Perfect Stream Protocol (PS1).
SRT.
Pro-MPEG.
RIST.

PS1 protocol¶

The PS1 protocol works on the principle of Automatic Repeat reQuest (ARQ). It has a low resource consumption and allows you to transmit streams with a high bit rate.

On transmitter - configurable in output. Only one copy is available for one stream. It is required to register logins for receivers in Peer. UDP listen port is set, must be unique for each stream.

At the receiver - configurable in input. The host and port of the transmitter are indicated, as well as the login and password.

Encryption of streams is available (Crypto protection), AES-128 is used. To enable encryption on both sides, enter Crypt Passphrase - a shared key.

During operation, the receiver (client) transmits its statistics data to the transmitter (server). This can be viewed in the Peers section by selecting a client from the list.

Stream latency and the ability to correct losses depends on the receiver (client) settings:

Round Trip Time - RTT, ms, default 300. Estimated delay (ping) in the channel. After starting the stream, the real RTT can be seen in the statistics (PS1 recovery delay).

Client Latency (RTT multiplexor) - multiplier to RTT, by default 10, which determines the stream delay on the sender’s buffer. Those. the default buffer delay is 3000 ms.

From the sender’s side there is a delay setting (buffer length) - Latency (ms). It should be greater than the delays specified by clients.

The ability of a protocol to compensate for losses is determined by the number of retransmission requests and depends on Client Latency (RTT multiplexor). Large losses lead to additional network traffic. To reduce latency, you should fine tune these parameters.

See client statistics for the correctness of the protocol. Counters PS1 recovery - if Not found, increase the sender’s buffer, Dublicates - increase RTT.

Since the connection is initiated from the side of the receiver, the transmitter requires authentication, the receivers are registered in the peers section. Login and password required.

SRT protocol¶

Open protocol designed by Haivision. Based on the UDT protocol. It has wide distribution and good packet loss performance.

Use cases:

Peer between two pstreamers. On transmitter - endpoint is configured as output, listen mode (default). For one stream only one such output can be configured. Multiple receivers can be connected in this mode. For authorization, it is required to register receivers logins in Peer. On receiver - endpoint is configured as input. The host and port of the transmitter, have to be set up as well as login and password. Streamer uses stream ID in format “login|password” to transmit login and password in SRT.
Peer between pstreamer and third-part SRT streamer. On transmitter you can set up srt client mode, switching off listen. SRT stream ID is entered in login field, if needed. For listen mode, the IP-address authorization is available - entered in login field in Peer. On receiver it is available to turn on listen mode, enter the SRT stream ID in the login field, also you can specify the host from which the connection is allowed.

Work in the Listener mode: receiving and transmitting a TV channel stream with the indication of the receiving port.

Ports for listen mode must be unique.

Encryption of streams is available (Crypto protection), AES-128 is used. To enable encryption on both sides, enter Crypt Passphrase - a shared key.

SRT protocol options match the description - https://github.com/Haivision/srt/blob/master/docs/API/API-socket-options.md

If the transmitter uses the listen mode (default), then the connection is initiated by the receiver, and the transmitter requires authentication, the receivers are registered in the peers section. Login and password required.

Pro-MPEG protocol¶

Pro-MPEG COP3 is described in the IEEE standard (https://ieeexplore.ieee.org/document/6738329) and is supported by a number of equipment. Advantages - low latency. Its disadvantage is high additional traffic (overhead), and it does not work well with large packet losses (more than 0.2%).

This protocol is based on RTP with the addition of 2 channels for FEC (Error Correction Code). Two FEC channels use ports port+2 and port+4, which should be taken into account when adding multiple streams to one host or multicast group.

At the sender, the stream of RTP packets is grouped into a matrix with Cols columns and Rows rows. Example for cols=8 and row=4 (default):

RTP01	RTP02	RTP03	RTP04	RTP05	RTP06	RTP07	RTP08	R1
RTP11	RTP12	RTP13	RTP14	RTP15	RTP16	RTP17	RTP18	R2
RTP21	RTP22	RTP23	RTP24	RTP25	RTP26	RTP27	RTP28	R3
RTP31	RTP32	RTP33	RTP34	RTP35	RTP36	RTP37	RTP38	R4
C1	C2	C3	C4	C5	C6	C7	C8

Rx and Cx packets form data for FEC in rows and columns. The smaller the matrix size, the better the loss correction ability, but the more additional traffic. In this example, there are 12 FEC packets for 32 RTP packets of the stream.

Encryption of streams (Crypto protection) is available, AES-128 is used, but this is not included in the standard, therefore compatibility with third-party software or hardware is not guaranteed.

There are non-standard protocol extensions:

Multiplexing - multiplexing of RTP channels through one UDP port. Can simplify network setup. Header XOR - obfuscation of the RTP header. Will complicate the definition of the type of traffic on the network.

RIST protocol¶

It is new open protocol. It is based on RTP/RTCP. It works as Automatic Repeat reQuest (ARQ) without ACK, only NACK, which provides high efficiency.

It uses unicast and multicast

Simple and Main profiles are implemented. Simple uses 2 udp ports in a row, the specified port must be even. Main uses only one RTP port with data multiplexing.

On transmitter - endpoint is configured as output. Receiver address and port are configured for unicast mode. For multicast mode, you need to set the network interface, through which the data will be transmitted. Also for multicast mode you can set up receivers authorization through IP address in Peer.

On receiver - endpoint is configured as input. For unicast mode, the receiver port (listen) is configured and the network interface is required. For multicast mode, only multicast group and port are set.

RIST supports multiple individual peers (addresses). You can set weight (more than 1), to activate load balancing mode on peers depending of weight.

If the transmitter uses multicast, then there can be many receivers. In this case, it is possible to authenticate receivers by IP address. To do this, enable authentication in the transmitter settings (disabled by default) and add the client to the peer list, enter the IP address in the login field.”

Other protocols¶

In addition to peer protocols for receiving and transmitting streams, others are available:

Protocol	Input	Output
UDP	Yes	Yes
RTP	Yes	Yes
TCP	Yes	No
HLS	Yes	Yes

UDP (Unicast or Multicast) - reception and transmission of MPEG-TS in a UDP packet, up to 7 TS packets per UDP packet.

RTP (Unicast or Multicast) is a standard RFC-based protocol. Reordered package recovery is supported.

TCP - receiving MPEG-TS in TCP connection, TCP client mode.

HLS - receiving and transmitting MPEG-TS over http or Apple’s HLS protocol. If receiving an adaptive playlist, a stream with a highest bitrate is selected.

Connection of third-party applications¶

To support other protocols that are not supported by the built-in tools, it is possible to receive and transmit a stream through third-party console applications. There is a separate std protocol for input and output for this. The MPEG-TS stream is received and transmitted through the operating system I/O stream.

The setting specifies the console application (absolute path), command line. You can also set environment variables.

For input, when configuring an external application, it is necessary to exclude the appearance of messages in the standard output, only in stderr.

For output it is possible to set packing up to 7 MPEG-TS packets.

Input stream requirements¶

Compliant with iso13818-1, Single Program (SPTS) or Multi Program Transport Stream (MPTS). Features of MPTS are described below, further settings are indicated for Single Program.”

At least one audio track is required.

Streams without video are supported, enabled by Radio mode.

Supported encoded streams, you need to enable Scrambled Stream.

For synchronization, the stream must have valid PCR marks. It is possible to use PTS / DTS for synchronization, but this is not recommended.

Stream settings¶

Set a unique name. Use Latin, numbers, _-. signs.

Set approximate bitrate (bitrate). By default, the maximum value is 24000 (in the next versions of the program, the bitrate limit can be increased). For this setting, it is important that the bit rate is not less than the actual bitrate. The actual speed (speed) can be clarified further in the flow statistics. Setting the bitrate to a more precise value will reduce the CPU usage.

Stream Timeout - total stream timeout. If during this time there is no valid input stream, then a complete restart is performed.

Pause - transfer stream, as well as all input and output to inactive state. By default, when adding a new stream as well as input and output they will be paused and inactive.

PCR Fix - correction of broken PCR labels in video streams, especially important when receiving low-quality HLS video streams.

The program checks the input stream with input for its validity. If the stream fails, then input is considered abnormal.

Check Interval - Interval to re-check the stream.

MPEG-TS filter options:

Clean All Unnecessary Data - Deletes all data except PAT, PMT, video and sound.

Remove SDT - Deletes SDT data (channel name, provider, etc.).

Remove EIT/EPG - Deletes EPG data.

Remove Teletext - Deletes teletext data.

Remove Subtitles - Deletes subtitles data.

Source reservation¶

Several input can be specified as a list, but only one is active. If input crashed, then an attempt is made to use the next one in the list, and so on in a circle.

If stream has Fallback Check enabled, then when the backup input (not the first one in the list) is running, a recheck will be performed higher in the list with the Check Interval interval. If, when rechecking, the stream is valid, then stream switches to it.

Since the order of input matters, it can be changed. If input is paused, then it will not be taken into account when working.

Filtering and modifying MPEG-TS¶

By default, the MPEG-TS stream is transmitted as is.

For each input the following MPEG-TS stream filtering options are available:

PID Accept - list of allowed pids. If empty, allow all except PID Reject.

PID Reject - list of prohibited pids. Takes precedence over PID Accept.

It is possible to change the pid. For this, the PID Old and PID New lists are introduced.

Mapping PIDs and Languages - remapping the language of audio tracks.

Default Language - assign the default language if there is no language for the sound track.

For stream you can assign new MPEG-TS data (SDT table):

MPEG-TS Network ID
Service Name
Provider Name
Language

MPTS streams¶

MPTS stream - MPRG-TS stream with several streams (services), each of them has unique number (PNR). It is used for DVB broadcasting.

Filtering options are not available for MPTS streams. Streams are passed as is.

Maximum bitrate 64000kbps.

The mosaic feature is disabled by default. It is not recommended to enable it on weak CPUs, it can add jitter.

Stream diagnostics displays data for all programs separately, as well as summary statistics.

Demultiplexor¶

of the demux type to the SPTS stream, select a source and a PNR service. If the source MPTS is active, then a list will be available when selecting the PNR, otherwise you need to set the PNR manually.

Multiplexor¶

Assembles MTPS stream from separate SPTS streams. To confgure:

Create MPTS stream.
Add input of muxer type. You can add some MPEG-TS metadata configuration. For most applications, the default parameters are suitable.
Add output of muxer type into source stream. Set up service name and provider name if needed. Select language in MPEG-TS settings if you use non-latin alphabet.
Repeat for all sources.

Multiplexor generates SDT, NIT and TDT/TOT (time marks) for stream. EIT (EPG) is taken from source streams. New PIDs are assigned.

OTT service¶

HTTP-based streams output - HLS and MPEGTS over http. HTTPS (SSL) is supported. Output is enabled on the OTT tab of Stream settings.

URLs for connections have format:

http://host:port/http/stream/login/password - login/password authorization http://host:port/http/stream/login - login (token) authorization http://host:port/http/stream/ - IP authorization

host and port are set in http server settings. stream - stream id. Not to be confused with the sequence number in streams list. ID is shown in stream stats page header, id is set at stream creation and never changes.

Similarly for HLS:

http://host:port/hls/stream/login/password http://host:port/hls/stream/login http://host:port/hls/stream/

Output URL templates and working status are shown on stream stats page. Unauthorized access is denied, all clients should be registered in peer.

Additional HLS parameters are available in URL (optional):

url?a=1&s=40&m=40&v=5

a - =0 relative path in play list, =1 absolute path (default).

s - play list duration (sec), default 40.

m - minimum play list duration (sec), default 40.

Maximum play list duration is 60 sec. If current chunk buffer size is less than minimum in request, error 404 will be raised. This is done so that HLS would start with a full chunk buffer on the server.

v - HLS protocol version for play list. default is 5. Version change may be required for some players.

Streams manipulations¶

Delete stream. To delete stream, go to the stream settings and press the button Delete stream.

Stream cloning. To clone stream go to the source stream settings and press the button Clone stream.

Sorting. To set up the sorting of streams, click the Sort button in the stream list window. After that, specify the desired order by dragging the streams up and down the list. To save the specified sort order, click the Save order button or Cancel to cancel the changes.

Filtering the list. To filter the list of streams, click on the button with the search icon and enter the filter string. To cancel the filtering, press the back arrow button.”

Group operations. In the stream list filtering mode, it is possible to select multiple streams by turning on the checkboxes in the left column. If any streams are selected, the Delete and Clone buttons for the selected streams become available.

Reports and diagnostics¶

Streams - displays data about all stream in the form of a table. Pause - available to admin and restricted admin roles.

Detailed statistics and reports are available for each stream.

Peers - list of active recipients (clients). Separate statistics are available for each.

Stream analysis¶

The streams analyzer measures and analyzes various MPEG-TS stream parameters, which allows you to evaluate the quality.

Input speed - the speed (bitrate) of the stream in kbps. It changes after filtering by PCR tags. It is displayed as a graph, which also displays the output bitrate after the synchronizer.

Raw data speed - data input speed according to specified protocol. Overhead - the protocol service data overhead in %.

CC errors - skipped packets (CC, discontinuity). Periodic counters and summary counter for stream uptime are shown. Also value history chart is shown.

Scrambled - encoded MPEGTS ES packets counter. If value is more than 0, there are decoding issues in protected channels.

Sync by - synchronization source, PCR by default.

PCR interval - interval between PCR marks. Not more than 50 ms is recommended.

PCR jitter - accuracy characteristic of output stream syncronization. Measured as the difference between PCR and real time.

Analyze PCR PMT Gap - enabled by special setting. Spread between PCR and PTS/DTS for each ES is analyzed. There is corresponding value history chart. Too much spread can cause problems for players with small sync buffers. Enabled by setting Analyze PAT/PMT/KF.

PAT interval and PMT interval - measured interval between PAT (PMT) tables. Recommended value is not more than 500 ms. Enabled by setting Analyze PAT/PMT/KF.

Key Frame interval - measured interval between key frames. Not more than 1 sec is recommended for players with random time access. It is enabled by setting Analyze PAT/PMT/KF.

PAT/KF interval - Measures the average interval between the beginning and end of the PAT/PMT/SPS/PPS/KF sequence. The playback speed of the players depends on this. It is measured by the beginning of KF in the stream, so the real start time of the player will be longer. Enabled by setting Analyze PAT/PMT/KF.

Enabling Analyze PAT/PMT/KF enables the flow analyzer in permanent mode, which leads to an increase in CPU load.

System Monitor¶

Control of the main parameters of the operating system.

Mosaic¶

The function of taking a screenshot from the input stream. Included separately for each stream.

If not required, in order to save resources, it can be completely disabled in the Settings/Server Settings section.

Administration¶

In the Settings/Web Console Access/Administrators List section, users are added to access the web interface - the built-in HTTP-server.

Users are assigned roles:

admin - full access.

restricted admin - settings are unavailable, it is possible to change pause state.

viewer - only view mode access.

Configuration backup¶

In order to make a backup copy of the settings, you need to save the contents of the folder /opt/pss/config.

To restore the settings, stop the service and replace the contents of the folder /opt/pss/config.

EPG/XMLTV import¶

EPG data is collected into the EPG Database from various sources:

EIT from received streams (SPTS and MPTS). Enabled by setting Stream: Extract EIT to EPG Database.
Import in XMLTV format from various external sources. Configured in Configuration/EPG/EPG Sources settings.

The storage time of EPG events configured by EPG storage period (days) setting.

Auto-clean database - Deletes programs for which there are no events.

The EPG section displays EPG sources and related data. For each channel (EPG Channels List), you can set:

Channel Name - the name that will be used in the export on the XMLTV server.
Time Zone - you can adjust the time zone if it was not tied to UTC when importing.
EPG Channel Sets - bind the channel to the Channel Set (see below).

EPG/XMLTV server¶

Used for publishing EPG in XMLTV format. Configured in Configuration/EPG/EPG Server.

EPG Source Name and EPG Source URL parameters set the approriate values that are placed in the header of the XMLTV document.

EPG in XMLTV format is accessible by links http://host:port/xmltv and http://host:port/xmltv.gz. You should configure client for EPG Server in EPG Server Clients.

Default XMLTV list is empty for client. You have to add channel group in EPG Channel Sets and link it to client and then assign the necessary channels from sources to this group.

EIT generator¶

EIT data from EPG Database can be generated in SPTS Stream. To do this, in the Stream setup, you need to set EPG Source ID and select EPG Channel ID. At the same time, SDT will be generated, even if it is not in the source. Set SDT Data correctly.

If this Stream is used in a multiplexer, then the Service Name can be redefined separately in the output/mixer setting.

Connection of external monitoring systems¶

To connect third-party monitoring systems, statistics data can be requested through the built-in http-server. The request format is as follows:

GET http://host:port/data/server - general server data.

GET http://host:port/data/stream - list of streams.

GET http://host:port/data/stream/id - stream data with the specified id, which can be obtained from the list of streams.

The data is displayed in xml format.

The authentication method is BASIC. No authentication is required when requesting from localhost.

Sample scripts for connecting monitoring systems (zabbix) are available in the installation package.

Getting the http-server port from the pss.json settings file:

PSS_DEFAULT_PORT=43971
HTTP_PORT=$(jq '."web-server"."bind-port"' /opt/pss/config/pss.json) || exit 1
test $HTTP_PORT = 'null' && HTTP_PORT=$PSS_DEFAULT_PORT

The zabbix_update.sh script includes an example of getting a list of streams and generating metrics for zabbix-agent. Metrics include flow rates, lost and encrypted packets.

The update_data.sh script includes an example of getting data. In order not to load the server with frequent requests, caching is done.

URI=http://127.0.0.1:${HTTP_PORT}${URI_PATH}
wget -q $URI -O $XML_FILE

mosaic images can be obtained by queries:

GET http://host:port/data/mosaic - xml list of streams with available images.

GET http://host:port/data/mosaic/id - compressed jpeg image with normalized size. id is taken from the list of streams.

GET http://host:port/data/mosaic/id/program - For MPTS stream. program is 0-based program index.

GET http://host:port/data/mosaic/id/orig - a picture in png format of the original size.

GET http://host:port/data/mosaic/id/program/orig - For MPTS stream. program is 0-based program index.

Export of logs¶

To enable logs to syslog, add the settings in the pss.properties file:

log-to-syslog=true
syslog-name=pss

Reload the pss service.

Access to data of the built-in database¶

To store various statistics and events data, there is a built-in database of the sqlite type, the pss.sqlite file.

The storage duration is set in the main server settings.

Data from the database is available via http requests

GET http://host:port/data/dbstat?tables=table_list&interval=10&t1=from&t2=to&agg=avg&digits=2

table_list - table1[,table2…].

interval - second, default 5, valid >=5.

t1 < 0, t2 < 0 - relative time from now.

t1 > 0, t2 > 0 - absolute epoch time.

agg - aggregate function [avg, min, max, sum] default avg.

digits - for float after point, default 2.

The data is returned as json.

The list of tables can be seen by opening the pss.sqlite file.

Programm optimization¶

If there are problems with a large CPU load or lack of memory in configuration with large number of streams, then you can optimize the settings.

You can disable the MPEG-TS filtering and processing features if you don’t need to. By default, the stream has the Clean All Unnecessary Data function enabled, disable it if there is no unwanted data in the stream. Disabling these features completely will remove the Original Media Information section from the Report.

You can completely disable or change Mosaic settings. A completely disabling is done in the server settings. You can disable it individually for each stream, or change the update interval with the Check Interval setting.

Queue overload errors for DBStat and DBEPG databases¶

Errors occur due to insufficient databases performance - slow storage is used or the system is overloaded.

Databases location is configured by data-dir parameter in configuration file pss.configuration

Possible solutions of the problem:

Moving database files to /tmp. The system memory will be used, requires an estimate of free memory and setting up the storage time of statistics (see server settings). When the system restarts, the data will be lost.
Reduce the statistics detalization - see dbstat-detail. 5 sec by default. Can be increased up to 20.
Locate database in memory - set the dbepg-memory option to true.

Configuration import and export¶

Using python script¶

The export and import of the configuration is implemented via .m3u playlist, done via a Python script.

Python ver 3 is required in path /usr/bin/python3.

Using web-interface¶

In the list of streams, clicking on the Playlist button opens a dialog for exporting streams to an m3u8 playlist. Only those streams that are currently displayed based on the applied list filters are exported.

Settings:

Host / IP - the name of the server or the address that will be used to generate the URL of the streams.
Protocols - types of protocols for export.
Login - account used for URLs with access credentials.
Use Display Names - use stream Display name in m3u8-file instead of Stream name.

The playlist is downloaded as an m3u8 file when you click on the Download button.

When you click on the Import Playlist button, the dialog switches to the playlist import mode. When importing streams, existing streams are not deleted. For all imported streams, the import time is recorded in the Note field.