LiveStream 5 Administration Guide

Valid List Entries

Each list group entry must match the criteria outlined by the type's official specification in order to match the data contained in the actual web requests.

LiveStream 5 simplifies this by automatically validating any new entry that's added, either manually or imported from a text file. The interface will either automatically reformat the entry for you, or let you know that there's a problem.

This section will also explain which part of a web request each type of entry is matching against so that you can determine the most effective way to target particular kinds of web traffic.

NOTE: LiveStream is purposefully ignorant of the protocol with regards to filtering. Your list entries will target HTTP and HTTPS websites alike.

Domains & URLs

Domain entries

The internet is simply a vast network of computers, each with a numeric internet Protocol (IP) Address. However, navigation by IP address would make connecting to websites a difficult process, because it is hard for people to remember long sequences of numbers. The Domain Name System aids navigation by associating easily-remembered names to those IP addresses, serving as a "phone book" for the internet.

The DNS infrastructure is based upon a hierarchy, or an upside-down "tree" of labels. A domain name usually consists of two or more labels separated by dots. Each label to the left specifies a subdivision, or subdomain of the domain above it. The DNS system implements a series of top-level domains, like com, net, org, edu, au, nz, uk etc.

When organisations register a domain name for their website, they get their names from the institutions that control the top-level domains. For example, Getbusi's domain is getbusi.com. The top level domain is com and the subdomain that is registered to Getbusi is getbusi. When reading domain names, always read from right to left. As you read left, each label after a dot is a subdomain (or child domain) of the label to its right.

Keep this hierarchy in mind when adding a domain entry to a list. If you target domains at too high of a level you will also target all of the subdomains of the domain you have specified.

Example

If you were to add an entry for edu.au, you would effectively target every educational institution in Australia because they are all subdomains of edu.au. In fact, this example might be very useful for unmetered content or a general Allow List.

URL entries

Understanding the different parts of a URL is crucial to ensure your list groups target exactly what you want and no more.

In its strictest technical definition Uniform Resource Locator (URL) contains all of the information a browser needs to access a document being served by a web server.

For example, the address: http://www.example.com/path/to/document/file.html contains the following information:

  • Protocol the server uses to host the document: http://
  • Name of the server hosting the document: www
  • Sub-domain to which the server belongs: example
  • Top level domain hosting the subdomain: com
  • Directory location of the document: /path/to/document
  • Name of the document: file.html

As you can see, the URL describes everything your browser needs to retrieve the document file.html from the server www in the domain example.com.

Example

There are many websites where the type of content changes drastically from URL to URL. Category filtering will generally handle this for you but, for the sake of this explanation, a good example of this is news websites.

Most online news sites are divided up into sections. News.com.au, for example, is split up by:

National / World / Finance / Sport / Entertainment / Lifestyle / Travel / Technology / Video

Each section has its own URL path meaning that you can add news.com.au/entertainment or news.com.au/video to a deny list whilst still allowing access to the other sections of the site.

IP addresses, ranges and subnets

As mentioned previously, every hostname on the the internet points back to an IP Address. In many cases, a particular company, domain or service will have its own reserved IP addresses, either on an individual basis or in a block.

These entries allow you to target requests based on their specific destination IP address, a range of IP addresses or by the CIDR subnet of the IP.

Web applications are rarely accessed explicitly by IP instead of hostname but many vendors document which servers their applications connect to by the IP information.

Example

If your organisation implicitly trusts web content from Apple Inc. you can target their entire assigned IP block using just one subnet entry: 17.0.0.0/8

Add this entry to your authentication bypass list will mean that all clients of the proxy can access any Apple service without restriction.

File types

File type entries may be used to target certain kinds of files available on the internet.

Rather then targeting files by their extension (e.g. .exe) LiveStream uses the Internet Media Type specification (formerly known as MIME types) which tends to be more accurate.

File type entries must be divided into the specification's required format: top-level type name / subtype name. There are currently nine registered top level types: application, audio, example, image, message, model, multipart, text, video.

Example

Podcasts are an increasingly popular entertainment medium and can use upwards of 50 MB per episode. Many desktop and mobile applications will automatically sync and download new podcast episodes whenever they are connected to the internet.

For a BYOD network where you can't necessarily control access to the applications themselves, it may be prudent to deny access to this type of file (mpeg audio). In which case your file type entry would simply be audio/mpeg.

Expressions

Expression entries allow you to target the contents of a URL for specific keywords and terms. If a client has a policy implementing expression-based filtering any address entered into the address bar of the browser will be parsed for a match against the list group.

Expression lists are a powerful filtering tool, but are indiscriminate. Since they are based on pattern matching, you must be careful in how you apply them. For example, you may wish to block eBay and eBay-related sites. If you create an expression list blocking the URLS which contain ebay, you will also block sites like rosebay, because it contains the string ebay.

There are three types of expression entries to choose from:

  • Contains — parses the entire URL for the specified expression. This can also be used to target specific non-standard web ports e.g. :8443
  • Ends with — parses the very end of every URL, working backwards. This can be used to target specific file extensions such as .zip, .doc or .exe.
  • Query contains — parses the query section of a URL. The query section (if present) of a URL is the part following a question mark. These are typically found in URLs of pages that implement forms. This can be used to target certain terms in search engine URLs.
Example

There are many Anonymous Proxy websites which bill themselves as an "unblocker". By adding a contains expression for unblocker to your relevant deny lists, you can block all of these websites with one entry.