An Overview of the Data-Loader Landscape: Comparative Performance Analysis · HackerPapers discussions

An Overview of the Data-Loader Landscape: Comparative Performance Analysis — mentioned by davidbuniat

Tue, 15 Nov 2022 19:25:02 GMT

Re: HF - we know them and admire their work (primarily, until very recently, focused on NLP, while we focus mostly on CV). As mentioned in the post, a large part of Deep Lake, including the Python-based dataloader and dataset format, is open source as well - https://github.com/activeloopai/deeplake. Likewise, we curate a list of large open source datasets here -> https://datasets.activeloop.ai/docs/ml/, but our main thing isn't aggregating datasets (focus for HF datasets), but rather providing people with a way to manage their data efficiently. That being said, all of the 125+ public datasets…

Paper and discussion history. A mention is not an endorsement.

An Overview of the Data-Loader Landscape: Comparative Performance Analysis — HN submission

Tue, 15 Nov 2022 16:01:13 GMT

Launch HN: Activeloop (YC S18) – Data lake for deep learning

Hi HN, I'm Davit, the CEO of Activeloop (https://activeloop.ai). We've made a “data lake” (industry jargon for a large data store with lots of heterogeneous data) that’s optimized for deep learning. Keeping your data in an AI-optimized format means you can ship AI models faster, without having to build complex data infrastructure for image, audio, and video data (check out our GitHub here: https://github.com/activeloopai/deeplake). Deep Lake stores complex data such as images, audio, videos, annotations/labels, and tabular data, in the form of tensors—a type of data structure used in linear…

64 points; 24 discussion replies on HN. Paper and source history. A submission is not an endorsement.