Perun: Difference between revisions
(add osm_update.py) |
(add youtube_downloader.py) |
||
| Line 12: | Line 12: | ||
== /home/user/osm_update.py == | == /home/user/osm_update.py == | ||
Fetches the newest <code>.osc.gz</code> daily changefiles from [https://planet.openstreetmap.org/ openstreetmap] for osm container into <code>/mnt/maps/tmp/</code>, to be processed by [[Osm#Data_updates]]. The latest already imported changefile is identified by its sequence number, and that written to <code>/mnt/maps/state.txt</code>. The state number therefore refers to which changefiles have already been downloaded. Whether they are in the database is indicated if they are removed from <code>/mnt/maps/tmp</code> or not. | Fetches the newest <code>.osc.gz</code> daily changefiles from [https://planet.openstreetmap.org/ openstreetmap] for osm container into <code>/mnt/maps/tmp/</code>, to be processed by [[Osm#Data_updates]]. The latest already imported changefile is identified by its sequence number, and that written to <code>/mnt/maps/state.txt</code>. The state number therefore refers to which changefiles have already been downloaded. Whether they are in the database is indicated if they are removed from <code>/mnt/maps/tmp</code> or not. | ||
== /home/user/youtube_downloader.py == | |||
For regularly downloading youtube videos to be displayed in [[Lada#Youtube]]. It can be called in several different modes for downloading a single video is 1080p smaller version, downloading an entire channel (in update mode it will then only download the new not-yet-downloaded videos of said channel), or downloading the playlists of a channel. Only the mode is specified in the commandline and the ids are then written to stdin, if they are a list, on multiple lines. | |||
Revision as of 11:44, 26 May 2022
LXC container, connected to the internet and used for downloading everything. It has readwrite access to relevant mountpoints and has wget, youtube-dlp, rsync and lftp as download software installed.
/video/lectures/updatefeeds.sh
Completely automatic ETHZ-video lecture recording downloader. It reads dir;url lines from urls.csv where dir is a directory under /video/lectures and link is a RSS feed of an ETHZ video lecture series. It fetches the RSS feed through the feed2exec python module, that converts every item to a csv line. Those are internally interpreted to extract the published date in %Y-%m-%d format and the video url is then downloaded to {dir}/%Y-%m-%d.mp4. It prints a last line reporting the work done: amount of feeds fetched, files detected (which are mentioned in the feeds but also already downloaded) and new files downloaded.
/software/git_clone.sh
Fetches archived git repositories, ran automatically.
/software/syncrepo-template.sh
Fetches, through rsync, archlinux and artixlinux (depending on $1) entire package repositories. Also fetches database files allowing completely functional package mirror operations, see Lada#Arch and artix package mirror.
/home/user/osm_update.py
Fetches the newest .osc.gz daily changefiles from openstreetmap for osm container into /mnt/maps/tmp/, to be processed by Osm#Data_updates. The latest already imported changefile is identified by its sequence number, and that written to /mnt/maps/state.txt. The state number therefore refers to which changefiles have already been downloaded. Whether they are in the database is indicated if they are removed from /mnt/maps/tmp or not.
/home/user/youtube_downloader.py
For regularly downloading youtube videos to be displayed in Lada#Youtube. It can be called in several different modes for downloading a single video is 1080p smaller version, downloading an entire channel (in update mode it will then only download the new not-yet-downloaded videos of said channel), or downloading the playlists of a channel. Only the mode is specified in the commandline and the ids are then written to stdin, if they are a list, on multiple lines.
