WIP: Article checker with svelte

.gitignore

@@ -1,5 +1,33 @@
 .dev/
-
+.vscode/
 *.pyc
 *.log
 __pycache__/
+
+
+
+## svelte:
+# Logs
+logs
+*.log
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+pnpm-debug.log*
+lerna-debug.log*
+
+node_modules
+dist
+dist-ssr
+*.local
+
+# Editor directories and files
+.vscode/*
+!.vscode/extensions.json
+.idea
+.DS_Store
+*.suo
+*.ntvs*
+*.njsproj
+*.sln
+*.sw?

docker-compose.yaml

@@ -60,16 +60,22 @@ services:
     volumes:
       - ${CONTAINER_DATA}:/app/containerdata # always set
       - ${CODE:-/dev/null}:/code # not set in prod, defaults to /dev/null
-      - ${XSOCK-/dev/null}:${XSOCK-/tmp/sock} # x11 socket, needed for gui
-      # - ${XAUTHORITY-/dev/null}:/home/auto_news/.Xauthority # xauth needed for authenticating to x11
     environment:
-      - DISPLAY=$DISPLAY # needed to let x11 apps know where to connect to
-
       - DEBUG=${DEBUG}
-      - CHECK=${CHECK}
-      - UPLOAD=${UPLOAD}
       - HEADLESS=${HEADLESS}
       - REDUCEDFETCH=${REDUCEDFETCH}
     entrypoint: ${ENTRYPOINT:-python3 runner.py} # by default launch workers as defined in the Dockerfile
     stdin_open: ${INTERACTIVE:-false} # docker run -i
     tty: ${INTERACTIVE:-false}        # docker run -t
+
+
+  news_check:
+    build: news_check
+    image: news_check:latest
+
+    volumes:
+      - ${CONTAINER_DATA}:/app/containerdata # always set
+      - ${CODE:-/dev/null}:/code # not set in prod, defaults to /dev/null
+
+    ports:
+      - 8080:80 # 80 inside container

misc/sample_config/nas_sync.config

@@ -0,0 +1,12 @@
+settings {
+   logfile    = "/tmp/lsyncd.log",
+   statusFile = "/tmp/lsyncd.status",
+   nodaemon   = true,
+}
+
+sync {
+   default.rsync,
+   source = "/sync/local_files",
+   target = "/sync/remote_files",
+   init = false,
+}

news_check/Dockerfile

@@ -0,0 +1,25 @@
+FROM node:18.8 as build-deps
+
+WORKDIR /app
+COPY client/package.json ./
+COPY client/package-lock.json ./
+COPY client/rollup.config.js ./
+COPY client/src ./src/
+RUN npm run build
+
+
+FROM python:latest
+ENV TZ Europe/Zurich
+
+RUN apt-get update && apt-get install -y postgresql
+
+RUN mkdir -p /app/news_check
+
+COPY requirements.txt /app/requirements.txt
+RUN python3 -m pip install -r /app/requirements.txt
+COPY --from=build-deps /app/public /app/news_check/public
+
+COPY app /app/news_check
+WORKDIR /app/news_check
+
+CMD gunicorn app:app -w 1 --threads 2 -b 0.0.0.0:80

news_check/client/.gitignore

@@ -0,0 +1,4 @@
+/node_modules/
+/public/build/
+
+.DS_Store

news_check/client/README.md

@@ -0,0 +1,107 @@
+# This repo is no longer maintained. Consider using `npm init vite` and selecting the `svelte` option or — if you want a full-fledged app framework and don't mind using pre-1.0 software — use [SvelteKit](https://kit.svelte.dev), the official application framework for Svelte.
+
+---
+
+# svelte app
+
+This is a project template for [Svelte](https://svelte.dev) apps. It lives at https://github.com/sveltejs/template.
+
+To create a new project based on this template using [degit](https://github.com/Rich-Harris/degit):
+
+```bash
+npx degit sveltejs/template svelte-app
+cd svelte-app
+```
+
+*Note that you will need to have [Node.js](https://nodejs.org) installed.*
+
+
+## Get started
+
+Install the dependencies...
+
+```bash
+cd svelte-app
+npm install
+```
+
+...then start [Rollup](https://rollupjs.org):
+
+```bash
+npm run dev
+```
+
+Navigate to [localhost:8080](http://localhost:8080). You should see your app running. Edit a component file in `src`, save it, and reload the page to see your changes.
+
+By default, the server will only respond to requests from localhost. To allow connections from other computers, edit the `sirv` commands in package.json to include the option `--host 0.0.0.0`.
+
+If you're using [Visual Studio Code](https://code.visualstudio.com/) we recommend installing the official extension [Svelte for VS Code](https://marketplace.visualstudio.com/items?itemName=svelte.svelte-vscode). If you are using other editors you may need to install a plugin in order to get syntax highlighting and intellisense.
+
+## Building and running in production mode
+
+To create an optimised version of the app:
+
+```bash
+npm run build
+```
+
+You can run the newly built app with `npm run start`. This uses [sirv](https://github.com/lukeed/sirv), which is included in your package.json's `dependencies` so that the app will work when you deploy to platforms like [Heroku](https://heroku.com).
+
+
+## Single-page app mode
+
+By default, sirv will only respond to requests that match files in `public`. This is to maximise compatibility with static fileservers, allowing you to deploy your app anywhere.
+
+If you're building a single-page app (SPA) with multiple routes, sirv needs to be able to respond to requests for *any* path. You can make it so by editing the `"start"` command in package.json:
+
+```js
+"start": "sirv public --single"
+```
+
+## Using TypeScript
+
+This template comes with a script to set up a TypeScript development environment, you can run it immediately after cloning the template with:
+
+```bash
+node scripts/setupTypeScript.js
+```
+
+Or remove the script via:
+
+```bash
+rm scripts/setupTypeScript.js
+```
+
+If you want to use `baseUrl` or `path` aliases within your `tsconfig`, you need to set up `@rollup/plugin-alias` to tell Rollup to resolve the aliases. For more info, see [this StackOverflow question](https://stackoverflow.com/questions/63427935/setup-tsconfig-path-in-svelte).
+
+## Deploying to the web
+
+### With [Vercel](https://vercel.com)
+
+Install `vercel` if you haven't already:
+
+```bash
+npm install -g vercel
+```
+
+Then, from within your project folder:
+
+```bash
+cd public
+vercel deploy --name my-project
+```
+
+### With [surge](https://surge.sh/)
+
+Install `surge` if you haven't already:
+
+```bash
+npm install -g surge
+```
+
+Then, from within your project folder:
+
+```bash
+npm run build
+surge public my-project.surge.sh
+```

news_check/client/package.json

@@ -0,0 +1,23 @@
+{
+  "name": "svelte-app",
+  "version": "1.0.0",
+  "private": true,
+  "scripts": {
+    "build": "rollup -c",
+    "dev": "rollup -c -w",
+    "start": "sirv public --no-clear"
+  },
+  "devDependencies": {
+    "@rollup/plugin-commonjs": "^17.0.0",
+    "@rollup/plugin-node-resolve": "^11.0.0",
+    "rollup": "^2.3.4",
+    "rollup-plugin-css-only": "^3.1.0",
+    "rollup-plugin-livereload": "^2.0.0",
+    "rollup-plugin-svelte": "^7.0.0",
+    "rollup-plugin-terser": "^7.0.0",
+    "svelte": "^3.0.0"
+  },
+  "dependencies": {
+    "sirv-cli": "^2.0.0"
+  }
+}

news_check/client/public/global.css

@@ -0,0 +1,63 @@
+html, body {
+	position: relative;
+	width: 100%;
+	height: 100%;
+}
+
+body {
+	color: #333;
+	margin: 0;
+	padding: 8px;
+	box-sizing: border-box;
+	font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen-Sans, Ubuntu, Cantarell, "Helvetica Neue", sans-serif;
+}
+
+a {
+	color: rgb(0,100,200);
+	text-decoration: none;
+}
+
+a:hover {
+	text-decoration: underline;
+}
+
+a:visited {
+	color: rgb(0,80,160);
+}
+
+label {
+	display: block;
+}
+
+input, button, select, textarea {
+	font-family: inherit;
+	font-size: inherit;
+	-webkit-padding: 0.4em 0;
+	padding: 0.4em;
+	margin: 0 0 0.5em 0;
+	box-sizing: border-box;
+	border: 1px solid #ccc;
+	border-radius: 2px;
+}
+
+input:disabled {
+	color: #ccc;
+}
+
+button {
+	color: #333;
+	background-color: #f4f4f4;
+	outline: none;
+}
+
+button:disabled {
+	color: #999;
+}
+
+button:not(:disabled):active {
+	background-color: #ddd;
+}
+
+button:focus {
+	border-color: #666;
+}

news_check/client/public/index.html

@@ -0,0 +1,26 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+	<meta charset='utf-8'>
+	<meta name='viewport' content='width=device-width,initial-scale=1'>
+
+	<title>Svelte app</title>
+
+	<link rel='icon' type='image/png' href='/favicon.png'>
+	<link rel='stylesheet' href='/global.css'>
+	<link rel='stylesheet' href='/build/bundle.css'>
+
+	<script defer src='/build/bundle.js'></script>
+
+	<link href="https://cdn.jsdelivr.net/npm/daisyui@2.24.0/dist/full.css" rel="stylesheet" type="text/css" />
+	<script src="https://cdn.tailwindcss.com"></script>
+
+	<script src="https://cdnjs.cloudflare.com/ajax/libs/pdf.js/2.0.943/pdf.min.js"></script>
+	<html data-theme="light"></html>
+
+</head>
+
+
+<body>
+</body>
+</html>

news_check/client/rollup.config.js

@@ -0,0 +1,76 @@
+import svelte from 'rollup-plugin-svelte';
+import commonjs from '@rollup/plugin-commonjs';
+import resolve from '@rollup/plugin-node-resolve';
+import livereload from 'rollup-plugin-livereload';
+import { terser } from 'rollup-plugin-terser';
+import css from 'rollup-plugin-css-only';
+
+const production = !process.env.ROLLUP_WATCH;
+
+function serve() {
+	let server;
+
+	function toExit() {
+		if (server) server.kill(0);
+	}
+
+	return {
+		writeBundle() {
+			if (server) return;
+			server = require('child_process').spawn('npm', ['run', 'start', '--', '--dev'], {
+				stdio: ['ignore', 'inherit', 'inherit'],
+				shell: true
+			});
+
+			process.on('SIGTERM', toExit);
+			process.on('exit', toExit);
+		}
+	};
+}
+
+export default {
+	input: 'src/main.js',
+	output: {
+		sourcemap: true,
+		format: 'iife',
+		name: 'app',
+		file: 'public/build/bundle.js'
+	},
+	plugins: [
+		svelte({
+			compilerOptions: {
+				// enable run-time checks when not in production
+				dev: !production
+			}
+		}),
+		// we'll extract any component CSS out into
+		// a separate file - better for performance
+		css({ output: 'bundle.css' }),
+
+		// If you have external dependencies installed from
+		// npm, you'll most likely need these plugins. In
+		// some cases you'll need additional configuration -
+		// consult the documentation for details:
+		// https://github.com/rollup/plugins/tree/master/packages/commonjs
+		resolve({
+			browser: true,
+			dedupe: ['svelte']
+		}),
+		commonjs(),
+
+		// In dev mode, call `npm run start` once
+		// the bundle has been generated
+		!production && serve(),
+
+		// Watch the `public` directory and refresh the
+		// browser on changes when not in production
+		!production && livereload('public'),
+
+		// If we're building for production (npm run build
+		// instead of npm run dev), minify
+		production && terser()
+	],
+	watch: {
+		clearScreen: false
+	}
+};

news_check/client/scripts/setupTypeScript.js

@@ -0,0 +1,121 @@
+// @ts-check
+
+/** This script modifies the project to support TS code in .svelte files like:
+
+  <script lang="ts">
+  	export let name: string;
+  </script>
+ 
+  As well as validating the code for CI.
+  */
+
+/**  To work on this script:
+  rm -rf test-template template && git clone sveltejs/template test-template && node scripts/setupTypeScript.js test-template
+*/
+
+const fs = require("fs")
+const path = require("path")
+const { argv } = require("process")
+
+const projectRoot = argv[2] || path.join(__dirname, "..")
+
+// Add deps to pkg.json
+const packageJSON = JSON.parse(fs.readFileSync(path.join(projectRoot, "package.json"), "utf8"))
+packageJSON.devDependencies = Object.assign(packageJSON.devDependencies, {
+  "svelte-check": "^2.0.0",
+  "svelte-preprocess": "^4.0.0",
+  "@rollup/plugin-typescript": "^8.0.0",
+  "typescript": "^4.0.0",
+  "tslib": "^2.0.0",
+  "@tsconfig/svelte": "^2.0.0"
+})
+
+// Add script for checking
+packageJSON.scripts = Object.assign(packageJSON.scripts, {
+  "check": "svelte-check --tsconfig ./tsconfig.json"
+})
+
+// Write the package JSON
+fs.writeFileSync(path.join(projectRoot, "package.json"), JSON.stringify(packageJSON, null, "  "))
+
+// mv src/main.js to main.ts - note, we need to edit rollup.config.js for this too
+const beforeMainJSPath = path.join(projectRoot, "src", "main.js")
+const afterMainTSPath = path.join(projectRoot, "src", "main.ts")
+fs.renameSync(beforeMainJSPath, afterMainTSPath)
+
+// Switch the app.svelte file to use TS
+const appSveltePath = path.join(projectRoot, "src", "App.svelte")
+let appFile = fs.readFileSync(appSveltePath, "utf8")
+appFile = appFile.replace("<script>", '<script lang="ts">')
+appFile = appFile.replace("export let name;", 'export let name: string;')
+fs.writeFileSync(appSveltePath, appFile)
+
+// Edit rollup config
+const rollupConfigPath = path.join(projectRoot, "rollup.config.js")
+let rollupConfig = fs.readFileSync(rollupConfigPath, "utf8")
+
+// Edit imports
+rollupConfig = rollupConfig.replace(`'rollup-plugin-terser';`, `'rollup-plugin-terser';
+import sveltePreprocess from 'svelte-preprocess';
+import typescript from '@rollup/plugin-typescript';`)
+
+// Replace name of entry point
+rollupConfig = rollupConfig.replace(`'src/main.js'`, `'src/main.ts'`)
+
+// Add preprocessor
+rollupConfig = rollupConfig.replace(
+  'compilerOptions:',
+  'preprocess: sveltePreprocess({ sourceMap: !production }),\n\t\t\tcompilerOptions:'
+);
+
+// Add TypeScript
+rollupConfig = rollupConfig.replace(
+  'commonjs(),',
+  'commonjs(),\n\t\ttypescript({\n\t\t\tsourceMap: !production,\n\t\t\tinlineSources: !production\n\t\t}),'
+);
+fs.writeFileSync(rollupConfigPath, rollupConfig)
+
+// Add TSConfig
+const tsconfig = `{
+  "extends": "@tsconfig/svelte/tsconfig.json",
+
+  "include": ["src/**/*"],
+  "exclude": ["node_modules/*", "__sapper__/*", "public/*"]
+}`
+const tsconfigPath =  path.join(projectRoot, "tsconfig.json")
+fs.writeFileSync(tsconfigPath, tsconfig)
+
+// Add global.d.ts
+const dtsPath =  path.join(projectRoot, "src", "global.d.ts")
+fs.writeFileSync(dtsPath, `/// <reference types="svelte" />`)
+
+// Delete this script, but not during testing
+if (!argv[2]) {
+  // Remove the script
+  fs.unlinkSync(path.join(__filename))
+
+  // Check for Mac's DS_store file, and if it's the only one left remove it
+  const remainingFiles = fs.readdirSync(path.join(__dirname))
+  if (remainingFiles.length === 1 && remainingFiles[0] === '.DS_store') {
+    fs.unlinkSync(path.join(__dirname, '.DS_store'))
+  }
+
+  // Check if the scripts folder is empty
+  if (fs.readdirSync(path.join(__dirname)).length === 0) {
+    // Remove the scripts folder
+    fs.rmdirSync(path.join(__dirname))
+  }
+}
+
+// Adds the extension recommendation
+fs.mkdirSync(path.join(projectRoot, ".vscode"), { recursive: true })
+fs.writeFileSync(path.join(projectRoot, ".vscode", "extensions.json"), `{
+  "recommendations": ["svelte.svelte-vscode"]
+}
+`)
+
+console.log("Converted to TypeScript.")
+
+if (fs.existsSync(path.join(projectRoot, "node_modules"))) {
+  console.log("\nYou will need to re-run your dependency manager to get started.")
+}

news_check/client/src/App.svelte

@@ -0,0 +1,17 @@
+<script>
+	import PDFView from './PDFView.svelte';
+	import ArticleStatus from './ArticleStatus.svelte';
+	import ArticleOperations from './ArticleOperations.svelte';
+</script>
+
+
+<div class="flex w-full h-full gap-5 p-5">
+	<div class="w-3/5"><PDFView/></div>
+	<div class="divider divider-horizontal"></div> 
+	<div class="w-2/5">
+		<ArticleStatus article_id={42}/>
+		<div class="divider divider-vertical"></div> 
+		<ArticleOperations/>
+	</div>
+</div>
+

news_check/client/src/ArticleOperations.svelte

@@ -0,0 +1,28 @@
+<div class="toast">
+	<div class="alert alert-info">
+	    <div>
+            <span>New message arrived.</span>
+        </div>
+    </div>
+</div>
+
+<div class="grid grid-cols-3 gap-4">
+    <div class="highlight">01</div>
+    <div class="highlight">01</div>
+    <div class="highlight">01</div>
+    <div class="highlight">01</div>
+    <div class="highlight">01</div>
+    <div class="highlight">01</div>
+    <div class="highlight">01</div>
+    <div class="highlight">01</div>
+    <div class="highlight">01</div>
+
+</div>
+<style>
+    .highlight {
+        background-color: #f5f5f5;
+        border-radius: 5px;
+        padding: 10px;
+        margin: 10px;
+    }
+</style>

news_check/client/src/ArticleStatus.svelte

@@ -0,0 +1,25 @@
+<script>
+    export let article_id;
+    const Article = (async () => {
+    const response = await fetch('/api/article/' + article_id + '/get')
+    return await response.json()
+	})()
+    console.log(Article)
+</script>
+
+<div class="mockup-window border bg-base-300">
+    <h1 class="center">Article overview</h1>
+    <ul tabindex="0" class="menu p-2 shadow bg-base-100 rounded-box w-52">
+        {#await Article}
+            <li>...waiting</li>
+        {:then data}
+            <li><a href="#">{data.value}</a></li>
+            <li><a href="#">Item 2</a></li>
+        {:catch error}
+            <li>An error occurred!</li>
+        {/await}
+    
+        
+    </ul>
+    
+</div>

news_check/client/src/PDFView.svelte

@@ -0,0 +1,69 @@
+<!-- 
+<script>
+	var myState = {
+		pdf: null,
+		currentPage: 1,
+		zoom: 1
+	}
+  
+	pdfjsLib.getDocument('test.pdf').then((pdf) => {
+  
+		myState.pdf = pdf;
+		render();
+
+	});
+
+	function render() {
+		myState.pdf.getPage(myState.currentPage).then((page) => {
+	  
+			var canvas = document.getElementById("pdf_renderer");
+			var ctx = canvas.getContext('2d');
+  
+			var viewport = page.getViewport(myState.zoom);
+
+			canvas.width = viewport.width;
+			canvas.height = viewport.height;
+	  
+			page.render({
+				canvasContext: ctx,
+				viewport: viewport
+			});
+		});
+	}
+
+</script>
+  -->
+<!-- <div id="my_pdf_viewer">
+    <div class="mockup-window border bg-base-300">
+        <div id="canvas_container" class="flex justify-center">
+            <canvas id="pdf_renderer"></canvas>
+        </div>
+    </div>
+        
+
+    <div id="navigation_controls">
+        <button id="go_previous">Previous</button>
+        <input id="current_page" value="1" type="number"/>
+        <button id="go_next">Next</button>
+    </div>
+
+    <div id="zoom_controls">  
+        <button id="zoom_in">+</button>
+        <button id="zoom_out">-</button>
+    </div>
+</div> -->
+
+<script>
+	let pdf_file = 'test.pdf';
+</script> 
+
+<div class="mockup-window border bg-base-300 h-full w-full">
+	<object class="pdf-view" data="{pdf_file}" title="Article PDF"> </object>
+</div>
+
+<style>
+	.pdf-view {
+		width: 100%;
+		height: 100%;
+	}
+</style>

news_check/client/src/main.js

@@ -0,0 +1,10 @@
+import App from './App.svelte';
+
+const app = new App({
+	target: document.body,
+	props: {
+		name: 'world'
+	}
+});
+
+export default app;

news_check/requirements.txt

@@ -0,0 +1,4 @@
+flask
+peewee
+markdown
+gunicorn

news_check/server/README.md

@@ -0,0 +1,10 @@
+# Svelte.js + Flask
+
+A super simple example of using Flask to serve a Svelte app and use it as a backend server.
+
+Run the following for development:
+
+- `python server.py` to start the Flask server.
+- `cd client; npm install; npm run autobuild` to automatically build and reload the Svelte frontend when it's changed.
+
+This example just queries the Flask server for a random number.

news_check/server/main.py

@@ -0,0 +1,37 @@
+from crypt import methods
+import json
+from flask import Flask, send_from_directory, jsonify
+import random
+
+app = Flask(__name__)
+
+
+###############################################################################
+# SVELTE BACKEND. Always send index.html and the requested js-files. (compiled by npm)
+@app.route("/") #index.html
+def base():
+    return send_from_directory('../client/public', 'index.html')
+@app.route("/<path:path>") #js-files
+def home(path):
+    return send_from_directory('../client/public', path)
+
+
+
+###############################################################################
+# API for news_check.
+
+@app.route("/api/article/<int:id>/get")
+def get_article(id):
+    res = {"value": id}
+    return jsonify(res)
+
+@app.route("/api/article/<int:id>/set", methods=['POST'])
+def set_article(id):
+    return str(random.randint(0, 100))
+
+
+
+
+
+if __name__ == "__main__":
+    app.run(debug=True)

news_check/server/package-lock.json

@@ -0,0 +1,3 @@
+{
+  "lockfileVersion": 1
+}