What do you write yourself when using dlt?
Write a Python job that reads records from a source and loads them into a destination. You decide which records it reads and which fields identify them. Plan where the code runs, who schedules it and who notices a failed run.
When should a second run replace, add or update rows?
Append adds more rows. Replace writes a fresh copy in the selected destination scope. Merge uses matching keys, such as an order ID, to apply changes. If you need a deletion to carry through a merge, check how your source signals it and which deletion behavior your chosen setup supports.
A normal incremental read can skip a row that simply disappeared. dlt documents explicit deletion markers for supported merge behavior.
What would you need to rerun the job elsewhere?
Keep the Python code, its dependency versions, the destination data and the progress record used to resume. dlt can restore pipeline state from its destination when the pipeline name, destination location and dataset match. Decide whether your next run should resume or build a fresh copy before switching.
What does the library cost to use?
The dlt library is Apache 2.0 licensed. Your runtime and storage have their own costs. dltHub also offers separate products with additional capabilities.
How would you check a merge on two pretend orders?
Use a sheet of paper with made-up IDs. Do not connect a database or run a job.
- Choose the matching fieldWrite order IDs 10 and 11 in the copy.
- Plan one changed recordChange the amount for ID 10. With merge keyed by ID, expect one updated row for ID 10.
- Plan a missing recordDecide how the source would signal deletion of ID 11. Write whether the destination should retain, flag or remove it.