←back to thread

Data Version Control

(dvc.org)
213 points shcheklein | 1 comments | 19 Oct 24 16:56 UTC | HN request time: 0.205s | source
Show context
jerednel ◴[19 Oct 24 18:49 UTC] No.41889752[source]▶
It's not super clear to me how this interacts with data. If I have am using ADLS to store delta tables, and I cannot pull prod to my local can I still use this? Is there a point if I can just look at delta log to switch between past versions?
replies(1): >>41889814 #
riedel ◴[19 Oct 24 18:57 UTC] No.41889814[source]▶
DVC is (at least as I use it) pretty much just git LFS with multiple backends (guess actually a more simple git annex). It further has some rather MLOps specific stuff. Is handy if you do versions model training with changing data on S3.
replies(3): >>41890760 #>>41890767 #>>41890837 #
1. matrss ◴[19 Oct 24 21:15 UTC] No.41890837[source]▶
Speaking of git-annex, there is another project called DataLad (https://www.datalad.org/), which has some overlap with DVC. It uses git-annex under the hood and is domain-agnostic, compared to the ML focus that DVC has.