Data Ingestion


OSDU Basics.

Ingesting and retrieving data from OSDU



Challenges

Manifest Ingestion

OSDU uses manifests to describe data to be ingested.

Objective

Submit a manifest for ingestion using the workflow trigger endpoint.

Example

curl -X POST http://challenge/api/workflow/v1/workflow/osdu/ingestion \
  -H "data-partition-id: opendes" \
  -H "Content-Type: application/json" \
  -d '{"executionContext": {"manifest": {"kind": "osdu:wks:Manifest:1.0.0"}}}'

Connect with SSH

Link your SSH key, then connect with: ssh hacker@dojo.idg.ctf

File Service

The File Service handles binary file uploads and downloads.

Objective

Get a signed URL to upload a file.

Example

curl -X POST http://challenge/api/file/v2/getFileUploadURL \
  -H "data-partition-id: opendes" \
  -H "Content-Type: application/json" \
  -d '{"fileName": "welldata.las"}'

Connect with SSH

Link your SSH key, then connect with: ssh hacker@dojo.idg.ctf

Workflow Service

The Workflow Service orchestrates data processing pipelines.

Objective

Check the status of a running workflow.

Example

curl -H "data-partition-id: opendes" \
  http://challenge/api/workflow/v1/workflow/osdu/ingestion/workflowRuns/run-12345

Connect with SSH

Link your SSH key, then connect with: ssh hacker@dojo.idg.ctf

30-Day Scoreboard:

This scoreboard reflects solves for challenges in this module after the module launched in this dojo.

7-Day | 30-Day | All-Time

Rank Hacker Badges Score