Data Engineer
What the work is
Designs, builds, operationalises and maintains the systems and processes for storing, transforming and analysing datasets.
Would you enjoy the work?
These are the main tasks in the official description. Mark how each one sounds to you.
- Builds, tests and maintains data pipelines to support analytics and data processing systems
- Develops and optimises processes and tools for data extraction, transformation and loading
- Identifies, designs and implements process improvements, including automating manual processing, optimising data delivery, and redesigning infrastructure for optimum scalability
- Implements secure data handling and storage procedures, protects data privacy, and ensures compliance with regulations and best practices
Try it
A simulation and a case study from this kind of work, set in the data team of a fictional online grocery delivery business. They are fictional.
The kind of days you could have
Good days
- A messy pile of numbers becomes one chart that makes everyone say 'oh, that's why'
- A tool you built saves the warehouse team a whole morning every week
- You catch a data error before it reaches the people who would have acted on it
Hard days
- Someone senior loves a chart that you know does not tell the whole story
- Hours of cleaning data that nobody will ever see, followed by a request to redo it
- A pipeline breaks at three in the morning and the dashboards go blank
Written by Career Calling as a general guide. Real roles vary by employer and place.
Work a mission
Step into a simulated workplace in this field, with a brief from your manager.
Description and tasks: Australian Bureau of Statistics, Occupation Standard Classification for Australia, 2024, Version 1.0 (opens another site), occupation 223233, used under a Creative Commons Attribution 4.0 licence. The skill guide above is our plain English summary.