HackerPapers

arXiv:1705.07962cs.LG

pix2code: Generating Code from a Graphical User Interface Screenshot

Tony Beltramelli

Abstract

Transforming a graphical user interface screenshot created by a designer into computer code is a typical task conducted by a developer in order to build customized software, websites, and mobile applications. In this paper, we show that deep learning methods can be leveraged to train a model end-to-end to automatically generate code from a single input image with over 77% of accuracy for three different platforms (i.e. iOS, Android and web-based technologies).

arXiv comments
Journal reference
DOI
Subjects
cs.LGMachine Learning cs.AIArtificial Intelligence cs.CLComputation and Language cs.CVComputer Vision and Pattern Recognition cs.NENeural and Evolutionary Computing

On Hacker News

Linked by 2 identifiable commenters in 2 comments across 2 threads, from Aug 2019 to Aug 2019. Found in 4 HN submissions from May 2017 to Sep 2017. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments2 threads

  1. AI startup claims to automate app making but actually just uses humans 1 comment

    • This is not true at all, and no offense, is a misunderstanding of how NNs can be used in other applications. For one, what you are modeling itself does not need to be differentiable, only the network itself needs to be. Second, using neural networks in combination with other…

      deepnotderpView on HNParent context

  2. Show HN: I trained a deep learning model to build sites for me 1 comment

    • It's really great to see more and more companies emerging in the market from these initial ideas/PoC: [1] Pix2Code: https://arxiv.org/abs/1705.07962 [2] Sketch2Code: https://airbnb.design/sketching-interfaces/ [3] ScreenShot2Code…

      ReDeiPiratiView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.