Structured Wikivoyage Exports

Small utility to convert the wikitext data from the Wikivoyage dumps into a structured format. The goal is to make it easier to work with the data and extract useful information programmatically.

Usage

Docker

This script is intended to be run with docker. A docker image is available from the GitHub registry. For example, you may run it using the filesystem handler with docker run -e HANDLER=filesystem -e HANDLER_FILESYSTEM_OUTPUT_DIR=/output -v ./output:/output --ulimit nofile=65536:65536 ghcr.io/bcye/structured-wikivoyage-exports. For all the different options, refer to the docs.

Types

TypeScript types for consuming the json output are available, you may install them from the @bcye/structured-wikivoyage-types npm package. Refer to the included docstrings in types/index.d.ts for reference.

Documentation

See docs for more information on how to use this utility.

Testing

Run PYTHONPATH=. pytest from inside the venv

License

Code

All code and documentation unless otherwise stated is licensed under the AGPLv3 license, refer to LICENSE for the full license text. The types package and all its code is licensed under MIT.

Examples

Files in the docs/example and tests/fixtures are copies (.txt) or derivatives (.json) of the Boston Article on Wikivoyage and licensed under CC BY-SA 4.0. A list of contributors is available on the original article.

1.7 KiB Raw Permalink Blame History