Hacker News

I gave a prompt and it stright up hallucinated. My prompt was about writing an article about the advantages and disadvantages of rust in the robotics ecosystem. It claimed that google cartographer was written in rust. The annoying thing about this is that it was quite convincing, I found the citation it used to be geeks for geeks blogspam that did not mention cartographer any where so I went and checked it was a C++ only project. Its worrisome when you see people relying on llms for knowledge.

Very cool! I asked it to create an article on the topic of my thesis and it was very good, but it lacked nuance and second order thinking i.e. here's the thing, what are the consequences of it and potential mitigations. It was able to pull existing thinking on a topic but not really synthesise a novel insight.

Synthesis of Topic Outlines through Retrieval and Multi-perspective Question Asking.

From the paper it seems like this is only marginally better than the benchmark approach they used to compare against:

>Outline-driven RAG (oRAG), which is identical to RAG in outline creation, but

>further searches additional information with section titles to generate the article section by section

It seems like the key ingredients are:

- generating questions

- addressing the topic from multiple perspectives

- querying similar wikipedia articles (A high quality RAG source for facts)

- breaking the problem down by first writing an outline.

Which we can all do at home and swap out the wikipedia articles with our own data sets.

"Sign in with Google" is a show-stopper.

Reminds me of Cuil.

> Cuil worked on an automated encyclopedia called Cpedia, built by algorithmically summarizing and clustering ideas on the web to create encyclopedia-like reports. Instead of displaying search results, Cuil would show Cpedia articles matching the searched terms.

https://en.wikipedia.org/wiki/Cuil

Does anyone have more info on this? They thank Azure at the top so I'm assuming it's a flavor of GPT? How do they prevent hallucinations? I am always cautious about asking an LLM for facts because half of the time it feels like it just adds whatever it wants. So I'm curious if they addressed that here or if this is just poorly thought-out...

Seems a promising approach. Feedback at the bottom is (?) missing a submit button. Article was fine, but veered into overly verbose with redundant sections. A simplification pass, even on the outline, could help.

We have been discussing a similar idea with friends.

The topic of knowledge synthesis is fascinating, especially in big organisations.

Moving away from fragmented documents into a set of facts from which LLM synthetize documents from, tailored for the reader.

There are few tricks that would be interesting to have working.

For instance the agent keep evaluating itself against a set of questions. Or user adding questions to see if the agent is able to understand the nuances of the topic and so if it can be trusted.

(Not dissimilar to what would be regression testing in classical software engineering)

Then the "homework" sections, when we ask human experts to evaluate that the facts stored by the agents are still relevant and up to date.

All these can then be enhanced with actions usable by the agent.

Think about fetching the PoC for a particular piece of software. It is the employer Foo.

If we write this down in a document, it will definitely get outdated when Foo move, or get promoted.

If we put this inside a knowledge synthesis system, the system itself may keep asking every 6 months to Foo if it is still the PoC for the software project.

Or it could daily talk with the LDPA system and ask the same question as soon as it notices that Foo has changed its position or reporting structure.

This can be expanded for processes to follow. Report to create, etc...

STORM motivated me to independently create my own similar project https://github.com/impredicative/newssurvey which works very differently to write a survey article on a medical or science topic. Its generated samples are linked in the readme.

I like this, but the quality is lower and more voluminous than phind or perplexity.

But I like the direction of the research. I'd like to be able to specify the output reduction prompts and to tweak the evaluation agents.

This is "just" multi-agent summarization and synthesis. Most summarizers are already doing this.

Nice thing that is this is open source, https://github.com/stanford-oval/storm

[1] https://www.phind.com/search?cache=z3qe9c0z6yb0x1hqbq64mrci

[2] https://www.perplexity.ai/search/please-summarze-and-explain...

I want to build this locally, I think it would be an absolute killer product. Could also consider doing an internet "deep dive" where the agent would browse for maybe 1-2 hours before sorting & collating the data. Add multi-modality for even more intelligence-gathering.

Fascinating. Last summer, inspired by AutoGPT, I made a simple Python script that does a web search for a query and uses that to answer the user's question. Looking at this I'm thinking, I could take the web results and ask it to reformat them in the style of Wikipedia, and wondering how that compares.

(I built it because ChatGPT couldn't search the web yet. When Phind launched a few weeks later, my project was basically obsolete!)

It seems the main improvement this paper has over that naive approach is the quality of the inputs, i.e. using "trusted sources" rather than random web results. (They appear to get their sources from Wikipedia itself?)

I'm not sure how much value all the other steps in the process add though.

I keep getting:

>Sorry, STORM cannot follow arbitrary instruction. Please input a topic you want to learn about. (Our input filtering uses OpenAI GPT-3.5, which may result in false positives. We apologize for any inconvenience.)

A very interesting project. Btw, I could not find a way to delete my account when created. I've also found that the generated report is very generic and quickly goes outside the actual question or specific theme/keywords used.

A final point, the notice states that "The risks associated with this study are minimal. Study data will be stored securely, in compliance with Stanford University standards, minimizing the risk of confidentiality breach." When I use STORM, I can see other people's request. Are they supposed to be confidential?

We use an approach inspired by this project to generate high level pages about city POIs such as this one: https://audiala.com/en/united-states/philadelphia/edgar-alla...

It's far from being perfect yet, sometimes too shallow and lacking a guiding thread, but after few iterations we believe it should offer all the information a visitor might need when planning a visit.

I feel like this is the opposite of what LLMs are useful for. I like using LLMs to summarize and get an immediate answer to a specific question, like the AI-generated summary in Google Search. In that case, the increase in convenience outweighs the decrease in reliability. But if I wanted to read a full article about a topic, I would no longer be concerned about convenience, so I would look for a more reliable source than an LLM.

Black text on dark grey background not the most ideal for main hero segment.

Also would love a way to try without google authentication

for the sources it used the crappy AI generated websites that pop up in the first page of google

Funny enough few months ago I was thinking how would Wikipedia written by AI look like. Imagine automating writing of knowledge so humans don't have to crawl the Web , books and papers to write knowledge articles.

Overview: https://storm-project.stanford.edu/research/storm/

It gets commented often, but:

   `Panther Moderns,' he said to the Hosaka, removing the trodes. `Five minute precis.' `Ready,' the computer said.

What's the point of the "elaborate on purpose" box? It makes you fill it in but doesn't seem to affect the article, at least not that I can tell.

I want to write a funding application, can you help me?

I want write a funding application, can you help me?

I kept tripping some sort of "don't give it direct instructions" filter, but it gave some interesting results. I asked it, multiple times, about my BitGrid (which it actually read about, and included!), FPGAs, LUTs and energy usage. It kept talking about the problems with the technology, and the need for specialist to program it, and environmental impacts.

I did discover a dearth of published information about just how much energy a 4x4 LUT requires per cycle, and it's idle power.

I don't know, remindes me of GPT o1:

Lots of text, lots of headers, but extremly shallow.

Did anyone figure out how to share the article after you generate it?

I've gone from thinking AI would only further degrade our sources of news and information to thinking perhaps AI is the only thing that can help combat misinformation.

Everything is so siloed & biased now, it's hard to find any presentation of a topic from a source that has no agenda. AI to help surface, aggregate, and summarize real data, expert opinion, and analysis like this would be really powerful and much needed. Essentially on-demand wikipedia articles held to the same editorial standards. Wikipedia isn't perfect by far, but their model has been surprisingly successful considering the challenge.

Doesn't even work. 500 error

Not to be confused with Storm the language https://storm-lang.org/index.html

I was impressed at the beginning, but then disappointed seeing hallucination in easily-verifiable historical information :(

[flagged]