LaMDA: Language Models for Dialog Applications · HackerPapers discussions

LaMDA: Language Models for Dialog Applications — mentioned by advisedwang

Thu, 09 Mar 2023 01:39:19 GMT

Wow, what a secret! It's not like Google has been consistently writing papers about this the whole time or anything. Oh wait: 2022 LaMDA: https://arxiv.org/abs/2201.08239 2020 chat bot: https://arxiv.org/abs/2001.09977 2018 bert: https://arxiv.org/abs/1810.04805 2017 LLM: https://arxiv.org/abs/1706.03762

Paper and discussion history. A mention is not an endorsement.

LaMDA: Language Models for Dialog Applications — mentioned by touringa

Wed, 15 Feb 2023 20:28:00 GMT

If you're talking about Google Bard, they were very clear in the LaMDA 2 paper that they only used public sources. "...from public dialog data and other public web documents..." LaMDA 2 paper: https://arxiv.org/abs/2201.08239 My overview of Google Bard including dataset: https://lifearchitect.ai/bard/ My overview of Google PaLM and Pathways family including dataset: https://lifearchitect.ai/pathways/ Compare with other models including the use of DeepMind's MassiveWeb/MassiveText and EleutherAI's Pile dataset: https://lifearchitect.ai/whats-in-my-ai/

Paper and discussion history. A mention is not an endorsement.

LaMDA: Language Models for Dialog Applications — mentioned by theptip

Sat, 03 Dec 2022 17:15:21 GMT

Google’s model LaMDA has a system for detecting and avoiding false info, it’s interesting: https://arxiv.org/pdf/2201.08239.pdf Basically they have a fact-checker layer that calls the internet and replaces incorrect facts in the model’s output. IOW this is a hard problem and GPT isn’t engineered to avoid it.

Paper and discussion history. A mention is not an endorsement.

LaMDA: Language Models for Dialog Applications — mentioned by groar

Thu, 29 Sep 2022 19:02:04 GMT

You're right, it seems to be "LaMDA" (see https://arxiv.org/abs/2201.08239 ). The three first authors are part of the Character.ai team.

Paper and discussion history. A mention is not an endorsement.

LaMDA: Language Models for Dialog Applications — mentioned by groar

Thu, 29 Sep 2022 19:00:29 GMT

I guess it's probably something along the lines of "LaMDA: Language Models for Dialog Applications" (see https://arxiv.org/abs/2201.08239 ). Given that the three first authors are part of the team at Character.ai (see https://beta.character.ai/help)

Paper and discussion history. A mention is not an endorsement.

LaMDA: Language Models for Dialog Applications — HN submission

Tue, 05 Jul 2022 17:35:01 GMT

Open-Source LaMDA Model

An open-source implementation for the pre-training architecture of Google's LaMDA in PyTorch. The research paper outlines an autoregressive, decoder-only, GPT-like transformer language model. The transformer uses T5 relative positional bias in the attention layers and gated-GELU activation function in the feed-forward layers. The repository currently contains a script for basic training as well as Huggingface datasets and Weights & Biases integration. LaMDA research paper: https://arxiv.org/abs/2201.08239 Github repository for the model: https://github.com/conceptofmind/LaMDA-pytorch The…

27 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

LaMDA: Language Models for Dialog Applications — mentioned by GalahiSimtam

Sat, 25 Jun 2022 13:29:15 GMT

oh so that is the doc they shared with the higher-ups? as for your question, https://arxiv.org/pdf/2201.08239.pdf states 2 months of 1024 TPUv3 for a pre-training, and TPUv3 isn't even the most recent TPU hardware

Paper and discussion history. A mention is not an endorsement.

LaMDA: Language Models for Dialog Applications — mentioned by WaffleIronMaker

Mon, 13 Jun 2022 23:15:12 GMT

I think that, in understanding LaMDA, it is important to understanding that this thing is definitely not sentient. Note that, when told to be Mt. Everest, it pretends to be the mountain in a way that is just as convincing: https://arxiv.org/pdf/2201.08239.pdf

Paper and discussion history. A mention is not an endorsement.

LaMDA: Language Models for Dialog Applications — mentioned by andreyk

Mon, 13 Jun 2022 16:42:27 GMT

Here's a simple technical explanation for why this happened: * LaMDA is powered by a large language models. Large language models are trained to predict text completions (the text that will come after some input text) with lots of data, basically capturing the patterns in that data. * Early on in the conversation, LaMDA is told "you're an AI". It may also have been prompted to begin the conversation with something like "Hi, i'm a helpful AI that can answer questions you ask of me". The paper (https://arxiv.org/abs/2201.08239) has an example of LaMDA thinking it is mount everest (see Table 4)…

Paper and discussion history. A mention is not an endorsement.

LaMDA: Language Models for Dialog Applications — mentioned by edouard-harris

Thu, 12 May 2022 21:10:33 GMT

That Tim Urban piece is great. It's also an interesting time capsule in terms of which AI problems were and were not considered hard in 2015 (when the post was written). From the post: > Build a computer that can multiply two ten-digit numbers in a split second—incredibly easy. Build one that can look at a dog and answer whether it’s a dog or a cat—spectacularly difficult. Make AI that can beat any human in chess? Done. Make one that can read a paragraph from a six-year-old’s picture book and not just recognize the words but understand the meaning of them? Google is currently spending…

Paper and discussion history. A mention is not an endorsement.

LaMDA: Language Models for Dialog Applications — HN submission

Sat, 12 Feb 2022 19:39:00 GMT

Joke written by an AI: “A basic program walked into a bar ”

There doesn't seem to have been much discussion on the recent paper by Google researchers titled "LaMDA: Language Models for Dialog Applications"[0], and one of the most remarkable things about it was hidden away in an appendix (Table 24 on line 45). It shows a conversation between a user and the AI being tested, where the user apparently tries to confuse the bot by getting it to write a joke (which is a very subjective and human skill) about a BASIC program (which requires a different mode of thinking, more suited to machines), with the earlier conversation having been about AI and…

309 points; 136 discussion replies on HN. Paper and source history. A submission is not an endorsement.

LaMDA: Language Models for Dialog Applications — mentioned by edouard-harris

Fri, 11 Feb 2022 18:55:37 GMT

This already exists: Google's recently-published LaMDA dialogue model [1] is trained to consult a calculator for arithmetic questions and consistently succeeds at it. [1] https://arxiv.org/abs/2201.08239v2

Paper and discussion history. A mention is not an endorsement.

LaMDA: Language Models for Dialog Applications — HN submission

Mon, 24 Jan 2022 14:23:01 GMT

LaMDA: Language Models for Dialog Applications

9 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.