Learning to Generate Reviews and Discovering Sentiment · HackerPapers discussions

Learning to Generate Reviews and Discovering Sentiment — mentioned by cmorez

Sun, 02 Jun 2024 04:51:22 GMT

> [...] but they certainly don't prove what OP claimed. OP's claim was not: "LLMs know whether text is true, false, reliable, or is epistemically calibrated". But rather: "[LLMs condition] on latents *ABOUT* truth, falsity, reliability, and calibration". > It's also very different to ask a model to evaluate the veracity of a nonsense statement, vs. avoiding the generation of a nonsense statement [...] probably could have been done with earlier generations of classifiers Yes. OP's point was not about generation, it was about representation (specifically conditioning on the representation of…

Paper and discussion history. A mention is not an endorsement.

Learning to Generate Reviews and Discovering Sentiment — mentioned by magoghm

Wed, 14 Feb 2024 14:24:52 GMT

I tried to find the where I heard that Radford was inspired by that blog post, but the closest thing I found is that in the "Sentiment Neuron" paper (Learning to Generate Reviews and Discovering Sentiment: https://arxiv.org/pdf/1704.01444.pdf), in the "Discussion and Future Work" section they mention this Karpathy paper from 2015: Visualizing and Understanding Recurrent Networks https://arxiv.org/abs/1506.02078

Paper and discussion history. A mention is not an endorsement.

Learning to Generate Reviews and Discovering Sentiment — mentioned by swyx

Wed, 06 Sep 2023 18:31:13 GMT

i highlighted this last night which seems to be making the rounds - https://twitter.com/swyx/status/1699369076529971545 Alec's CV (https://www.linkedin.com/in/alecradford/) seems to be: - 2011-2016 BSc Eng from Olin College - 2013+ started a data/AI consultancy as a sophomore that turned into a vague startup/product? - 2015 first paper on GANs coauthored with soumith https://arxiv.org/pdf/1511.06434.pdf%C3 - 2016 first GAN paper under openai email https://proceedings.neurips.cc/paper_files/paper/2016/file/8... - 2017 the generative reviews paper mentioend in the Wired article, with Ilya…

Paper and discussion history. A mention is not an endorsement.

Learning to Generate Reviews and Discovering Sentiment — mentioned by tensor

Tue, 23 May 2017 05:05:42 GMT

Sure, in looking at what it would cost to replicate this: https://arxiv.org/pdf/1704.01444.pdf They say they used four Titan X cards for a month. I was actually looking at google cloud machine learning since I thought being targeted to tensorflow would be the most cost effective. But it's 0.49/hour per ML training unit, or 1.47/hour per GPU. A basic gpu gives 3 training units, so I thought that something approximately similar would require 12 training units, which comes to something like $4000+ a month. Maybe I completely misunderstood the resources being offered though, because you are right…

Paper and discussion history. A mention is not an endorsement.

Learning to Generate Reviews and Discovering Sentiment — mentioned by tshadley

Fri, 07 Apr 2017 00:35:25 GMT

"The selected model reaches 1.12 bits per byte." (https://arxiv.org/pdf/1704.01444.pdf) For context, Claude Shannon found that humans could model English text with an entropy of 0.6 to 1.3 bits per character (http://languagelog.ldc.upenn.edu/myl/Shannon1950.pdf)

Paper and discussion history. A mention is not an endorsement.