HackerPapers

arXiv:2201.07725cs.CL

Data-to-Value: An Evaluation-First Methodology for Natural Language Projects

Jochen L. Leidner

Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.

Abstract

Big data, i.e. collecting, storing and processing of data at scale, has recently been possible due to the arrival of clusters of commodity computers powered by application-level distributed parallel operating systems like HDFS/Hadoop/Spark, and such infrastructures have revolutionized data mining at scale. For data mining project to succeed more consistently, some methodologies were developed (e.g. CRISP-DM, SEMMA, KDD), but these do not account for (1) very large scales of processing, (2) dealing with textual (unstructured) data (i.e. Natural Language Processing (NLP, "text analytics"), and (3) non-technical considerations (e.g. legal, ethical, project managerial aspects). To address these shortcomings, a new methodology, called "Data to Value" (D2V), is introduced, which is guided by a detailed catalog of questions in order to avoid a disconnect of big data text analytics project team with the topic when facing rather abstract box-and-arrow diagrams commonly associated with methodologies.

arXiv comments
9 pages, 6 figures, 4 tables
Journal reference
DOI
Subjects
cs.CLComputation and Language stat.MEMethodology (Statistics)

On Hacker News

Linked by 1 identifiable commenter in 2 comments across 2 threads, from Jul 2023 to Feb 2025. Found in 1 HN submission from Jan 2022 to Jan 2022. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments2 threads

  1. The time of evaluation driven development 1 comment

    • What is called "Evaluation-Driven Development" (EDD), I called "evaluation first" approach: https://arxiv.org/abs/2201.07725 Thinking about evaluation at the beginning, not at the end of a project, means that you don't fly blind during most of the project's execution duration.

      jll29View on HNParent context

  2. Why software projects take longer than you think: a statistical model (2019) 1 comment

    • I know a plan is good when my bottom-up estimates (padded with a buffer for unforeseen contingencies) align with my top-down gut instinct of how long a project of a certain complexity "should take". As an experienced professional, one often compares with past projects and looks…

      jll29View on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.