←back to thread

36 points dataflowmapper | 1 comments | | HN request time: 0.225s | source

Hi all,

I’m curious how folks handle the prep work for data imports/uploads into systems like Salesforce, Workday, NetSuite, or really any app that uses template based import for data loading, migration, or implementation.

Specifically: - How do you manage conversions/transformations like formatting dates, getting everything aligned with the templates, mapping old codes to new ones, etc.

- Are you primarily using Excel, custom scripts, Power Query or something else?

- What are the most tedious/painful parts of this process and what have you found that works?

Really appreciate any insights and am curious to learn from everyone's experience.

1. ensemblehq ◴[] No.43990869[source]
Through many different companies, we’ve primarily handled this with validation/transformation scripts across a variety of tools. What’s useful is attaching observability/logging and having each transformation step do one thing only (borrowing from the Unix philosophy). Having a good understanding of the schema designs for each system is key - at some organizations, we’ve codified the schemas. Pydantic models (and similar) have been very useful as well.