remoll/coss_archiving
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Switched to docker compose and wasted hours trying to have standalone firefox

Browse Source
This commit is contained in:
Remy Moll
2022-05-29 18:29:31 +02:00
parent 878a1dff5d
commit 54760abee4
13 changed files with 154 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
README.md
View File

@@ -3,7 +3,8 @@
A utility to fetch article requests from slack and generate pdfs for them, fully automatically.
## Running
## Running - Pure docker
> I recommend running with docker compose instead
### How to run - auto archiving mode
In this mode the program is launched as a docker container, in a headless mode. For persistence purposes a local storage volume is required, but that's it!
@@ -15,6 +16,12 @@ You can specify additional parameters:
`docker run -it -v <your storage>:/app/file_storage/ auto_news upload` catches up on incomplete uploads to archive.
`docker run -it -v <your storage>:/app/file_storage/ auto_news reducedfetch` makes assumption about the status of the slack chat and greatly reduces the number of api calls (faster start up).
These parameters can be combined (mostyl for testing I guess)
Finally for manual file verification:
`docker run -it -v <your storage>:/app/file_storage/ -e DISPLAY=":0" --network host -v \$XAUTHORITY:/root/.Xauthority auto_news check` lets you visually verify the downloaded files. The additional parameters are required in order to open guis on the host.
@@ -24,33 +31,51 @@ In this mode, a docker container is launched with an additional volume, the loca
`docker run -it -v <your storage>:/app/file_storage/ -v <your code>:/code/ --entry-point /bin/bash auto_news`
You are droppped into a bash shell, in which you can navigate to the `/code` directory and then test live.
### Cheat-sheet Remy:
`docker run -it -v /mnt/Data/COSS/Downloads/auto_news.container/:/app/file_storage/ auto_news`
`docker run -it -v /mnt/Data/COSS/Downloads/auto_news.container/:/app/file_storage/ -v /mnt/Data/COSS/Development/auto_news/app:/code --entrypoint /bin/bash auto_news`
`docker run -it -v /mnt/Data/COSS/Downloads/auto_news.container/:/app/file_storage/ -e DISPLAY=":0" --network host -v XAUTHORITY:/root/.Xauthority auto_news check`
## Running - Docker compose
I also wrote a rudimentary docker compose file which makes running much more simple. Just run
`docker compose --env-file <desired mode> up`
All relevant passthroughs and mounts are specified through the env-file, for which I configured 4 versions: production, debug (development in general), upload and check. These files will have to be adapted to your individual setup but can be reused more easily.
> Note:
>
> The `debug` requires additional input. Once `docker compose up` is running, in a new session run `docker compose --env-file env/debug exec bash`. The live-mounted code is then under `/code`. Note that the `DEBUG=true` environment variable is still set. If you want to test things on production, run `export DEBUG=false`.
## Building
### Things to keep in mind
The software (firefox, selenium, python) changes frequently. For non-breaking changes it is useful to regularly clean build the docker image! This is also crucial to update the code itself.
> The software (firefox, selenium, python) changes frequently. For non-breaking changes it is useful to regularly clean build the docker image! This is also crucial to update the code itself.
In docker, simply run:
`docker build -t auto_news --no-cache .`
where the `Dockerfile` has to be in the working directory
In docker compose, run the usual command, but append
`docker compose ... up --build`
## Cheat-sheet Remy:
`docker run -it -v /mnt/Data/COSS/CONTAINERDATA/:/app/file_storage/ auto_news`
`docker run -it -v /mnt/Data/COSS/CONTAINERDATA/:/app/file_storage/ -v /mnt/Data/COSS/auto_news/app:/code --entrypoint /bin/bash auto_news`
`docker run -it -v /mnt/Data/COSS/CONTAINERDATA/:/app/file_storage/ -e DISPLAY=":0" --network host -v XAUTHORITY:/root/.Xauthority auto_news check`
## Roadmap:
[ ] automatically upload files to NAS
[ ] handle paywalled sites like faz, spiegel, .. through their dedicated edu-sites
[ ] handle paywalled sites like faz, spiegel, .. through their dedicated edu-friendly sites
...