Skip to content

Home

Jump to bottom
Adrian Edwards edited this page Feb 24, 2024 · 12 revisions

data-ingest

Welcome!

This is the scraping code for the CampusPulse event aggregation scraper

It's easy to get up and running.

  1. Find an issue you'd like to help out with and add a comment saying that you're going to take it on.

  2. Set up a development environment.

  3. Read up on how our pipeline works.

  4. Send a pull request when you've got something ready for review.

  5. Pick another open issue to work on.

Contributing

  1. Find an issue you'd like to help out with
  2. Set up a development environment
  3. Read up on how our pipeline works
  4. Run the pipeline locally

Shared tooling

Some pre-built tools to help fetch or parse common data types

  1. Parse .json to .ndjson
Clone this wiki locally