Lada

LXC container

Webserver contents

Scripts in php display youtube, videos (TV series), movies, dlw and more. Video files are usually directly served but the php scripts always link to them as vlc://[lada https domain]/path/to/movie.mkv to be handled on the client-side browser in a specific way, for example opening vlc and streaming the https file over the network directly 'on-screen'.

Uploadable storage

/ram.php shows downloadeable files and provides an uploading form, /ramulpoad.php handes form and writes uploaded files to /var/www/html/ram/{filename}. Under /var/www/html/ram/ there is a tmpfs RAM filesystem mounted, to allow volatile operations, and therefore erasure across server reboots. It is mounted at boot by the systemd unit uploader in /etc/systemd/system/uploader.service.

Map

Makes cartography user-available in an interactive worldmap. /leaflet.html handles display of the maps from raster tiles and manages all the layers. It uses the leaflet javascript library, which is statically available in /leaflet/. Rendered map layers are specified by a tile URL in the format /osm/{layername}/{z}/{x}/{y}.png and any tile request will then be redirected to osm. Static raster tiles are directly mounted in the webserver, this is the case for one layer, /satellite/{z}/{x}/{y}.jpg. /leaflet.html also needs /leaflet.tilelayer.fallback.js[1] for fallback-enabled layers, this is currently used for the /satellite/ layer as it does not have all tiles available. Instead of displaying the error tile whenever a specific tile is missing (/satellite/ocean.jpg for /satellite/), it reuses upper-zoom tiles and pixellates them.

Stats

Read monitoring and display it with Javascript library plotly. Plotly.js is statically loaded from the webserver to allow completely internet-less operation. The data is in json prepended with "{globa_var_name}=" so that it is valid javascript and can be simply used in HTML:

<script src="/stats/{var_name}.latest.js"></script>

The plotly library then shows an interactive graph that allows zooming, panning and exporting view to .png.

Variations

Some data traces are available in reduced size, as in the time intervals are longer and the entire data spans a shorter time, usually under {var_name}.reduced.js. This is available in the graph as "{var_name} reduced".

Also, larger data traces are by default cutoff in time and use {var_name}.latest.js. To get the full data trace which usually loads multiple megabytes (and 10MB+ uncompressed data), add the ?mode=full url parameter to switch to {var_name}.data.js.

Arch and artix package mirror

Makes software zfs dataset available, all static files allow for package mirror operations. Use

Server = https://lada.tizarne.com/artix/$repo/os/$arch

for artix and

Server = https://lada.tizarne.com/arch/$repo/os/$arch

for arch in /etc/pacman.d/mirrorlist. To make the two directories available in the webserver root, symlinks are used :

arch -> data/software/arch                      
artix -> data/software/artix

Feeds (RSS)

Under /feeds, local podcast and radio recordings are served as RSS.

Piwigo

PHP picture gallery which is fully under /var/www/html/piwigo. It also uses a #mysql database to store all data. The pictures themselves are accessed under /var/www/html/piwigo/galleries/ and a symlink goes to family -> '/mnt/family/photos et vidéos/'. Piwigo has strong requirements for file- and directory names, which are enforced if /mnt/family/photos et vidéos/sanitize_names.py is run. This includes:

• no spaces
• no diatrics (é,è,à,ô,š,ť,ü,ö,...)
• no symbols except - and _ (?,!,+,=,@,#,$,%,&,*,),(,...)

Gitea

Go server for managing git repositories. but here it is used mainly to display archived repositories and allow browsing them in the browser link github, with syntax highlighting and markdown display. It listens at 10.0.3.2:7000, which is redirected by rigel like a few other open ports of lada. It uses the main postgresql database cluster and has the dbname=gitea for its data. It runs under the git user and manages repositories stored in /var/lib/gitea/data/gitea-repositories/. Because it expects to manage repositories instead of readonly displaying them, some workarounds are needed to add an archive repository:

• create a repository as the archives user in the webinterface
• remove the corresponding /var/lib/gitea/data/gitea-repositories/archives/{name}.git directory (note any CamelCase names are converted to lowercase)
• symlink to the actual repository, usually in /mnt/software/git_cloned/{author}/{NaMe}
• fetch the repository through the webinterface, it will give an HTTP error 500 but any subsequent refresh should display the repository.

Currently this process is a bash oneliner saved in the git user's bash_history:

for i in /mnt/software/git_cloned/*/*;do dest=/var/lib/gitea/data/gitea-repositories/archives/"$([ "${i}" = "${i/archlinux/}" ]|| echo 'aur_')$(basename "$i").git";dest="$(echo "$dest"|python -c 'print(input().lower())')";if ! [ -h "$dest" ];then ln -s "$i" "$dest";fi;done

Note that any archlinux (AUR) repositories are prepended with aur_{name} for clarity, and also note the use of python for lowercase-ing.

PHP scripts displaying data as user-accessible

These scripts may connect to a database and generate a HTML webpage to display video or audio data for display in a browser or as xml. This data is mostly readonly and for consumption, though the database may sometimes be written to regarding metadata like watching progress.

TV series

/videos is a forced-php script that tracks watching progress in the database under dbname=video. It displays all series on the landing page and makes a "watch next" vlc:// link available for each series. Some series that do not have ordered seasons will not have a global natch-next link, but on opening the series all seasons are shown and thos will have "watch next" links if the episodes are ordered. Unordered episodes means that in a season all episodes displayed have a green tick in the upper right corner if they have been watched. The entire script is server-side only and does not use javascript. It also features language track selection support by passing the ?audiolanguage={lang} URL parameter in the vlc:// links and title specification through the URL parameter &title={urlencode(title)} to be displayed in vlc. The vlc opening script also needs to carefully parse URL parameters and only take those that refer to it, like audiolanguage and title. This is because the next, next_season and watch URL parameters specify the mechanism to count episode progress. This takes place when vlc fetches the URL: it will be to the /videos script itself with some $_GET parameters. This script will then update the database accordingly and only after that set the HTTP header 302 redirect to point to the video file for vlc to start playing. This also allows for vlc to play the next track whenever the user presses "next", because the vlc playlist has only one item, it will just refetch the same url. But because the php script has previously updated the database, it will again update the watching progress, and then redirect to the next video file in the sequence.

Podcast feeds

/feeds is a forced-php script that connects to the database under dbname=audio. It generates podcast xml feeds from the database. The audio files are also assumed to be under /data/audio/podcasts/{db_audio:podcasts:directory}/{db_audio:episodes:filename}.

Forced php scripts

For cosmetic reasons and url shortness, some php scripts do not have a .php extension. They are then manually forced to be php-interpreted in the server. This also allows to reference nonexisting subdirectories "under" the script: /youtube is forced-php, and then /youtube/video/{videoid} and /youtube/channel/{channelid} are valid urls (that are also in practical use). Apache2 server configuration in /etc/apache2/apache2.conf:

<FilesMatch "youtube$|redirect$|feeds$|movies$|dvd$|videos$">
ForceType application/x-httpd-php
SetHandler application/x-httpd-php
</FilesMatch>

The FilesMatch section accepts regexes, so the a|b|c syntax allows a shorthand for repeated <File a>...</File><File b>...</File>... sections. This can become a security issue if the regex is edited to include wider syntax and more complicated regex parsing. A $ was added to force extension-less files only, as /style/youtube.css would have matched and been php-interpreted.

Databases

Normal postgresql database

Is started by systemd postgres service and keeps data in /pgsql/data/. Contains two dbnames videos and audio.

db videos

Biggest table is videos which contains all youtube videos and their descriptions which contribute most to the size. Tables playlists, channels and others provide all youtube functionality for three types of resources: videos, channels and playlists.

Also contains all videos data in tables series, seasons and episodes. They are only semi-automatically filled up, mostly through python. This happens through snippets in the history under /home/user/.python_history.

db audio

Contains data for /feeds in two tables: episodes and podcasts. Most items actually are podcasts but some radio recordings are also present as podcasts and are available as RSS feeds.

RAMdb

Volatile database, keeps its data in /mnt/ramdb/ which also is a 2GB tmpfs RAMdisk. This database has a dbname sponsorblock and a table sponsorblock that holds sponsorTimes.csv data for database-access instead of forcing the weberver to read a 1GB .csv with php. The volatility though is mostly for backup purposes: to avoid redundancy. The source of truth for that data is from the internet, but locally it is the .csv. Because that file is located in /youtube, it is backed up. The database would then only be backed up for the schema. The sponsorblock table was in the normal database before, but it changes so fast that the about-biweekly backup of the entire container ballooned from about 2.5GB up to 16GB per snapshot, and 25+GB for multiple snapshots. Now only the /mnt/ramdb_init/ data is backed up, about 20MB and generally static.

/mnt/ramdb_init

Contains empty database instance for copying over to the tmpfs upon container boot. It is a database with the sponsorblock table and its indexes, just with 0 rows. The configuration also states that the running port is 5433 to prevent conflict with the normal database (on default port 5432). On startup, the ramdb systemd service starts /root/ramdb.sh which sets the entire RAM database up and populates it by running /home/user/addcsv.py as user in a second thread.

mysql

Used by #Piwigo because it doesn't support postgresql. Listens on localhost:3306.

External redirects

Main website entry point and redirects some subdirectories to other webservers

• /osm to [osm]:80/
• /kiwix to localhost:8000/
• /dewa to [dewa]:8051/