R to @sensiblehq: this is awesome! been waiting for this

We’re excited to launch Sensible’s in-app dashboard. Now, you can track all your document extraction metrics in one place, in real-time. https://www.sensible.so/blog/sensible-dashboard

R to @DrJimFan: Seems like a huge opportunity for @sensiblehq

How can you measure LLM extraction accuracy when traditional ML confidence scores don’t work? Sensible’s confidence signals help to identify common sources of extraction uncertainty, and enhance your LLM prompting to achieve better results. https://www.sensible.so/blog/confidence-signals?utm_campaign=Confidence%20Signals&utm_source=twitter&utm_medium=social

R to @zachweinberg: 👇 https://www.sensible.so/

R to @sensiblehq: End/ If you liked this thread, please like and retweet the first post here: 👇

R to @sensiblehq: 9/ For an in-depth, check out the full post: https://www.sensible.so/learn/llm-document-extraction?utm_campaign=education&utm_source=twitter&utm_medium=social

R to @sensiblehq: 8/ Our goal is to bridge the gap between documents and software. These best practices have helped us scale our document question answering across varied layouts. Any questions, ask away!

R to @sensiblehq: 7/ CONFIDENCE SCORES: As LLMs lack a natural confidence metric, we developed confidence signals to flag potential uncertainties in prompts and answers for improved accuracy.

R to @sensiblehq: 6/ COST OPTIMIZATION: Reducing the context you feed the model saves money. Optimal chunking is crucial. We also use question collation and optimize table extraction to reduce costs.

R to @sensiblehq: 5/ LAYOUT PRESERVATION: GPT models are not intrinsically layout-aware. We embed layout info into the GPT context string via intelligent bounding box to whitespace conversion. This is most impactful with semi-structured documents with label/value pairs, tables, etc.

R to @sensiblehq: 4/ Context descriptions help situate the chunk text in a real-world scenario, leading to higher quality answers. Page hinting, which includes information about the position of text within pages, makes our chunk scoring position-aware.

R to @sensiblehq: 3/ CHUNKING: We divide documents into "chunks", calculate a similarity score between those chunks and your query, and use the highest-scoring chunks as context. We improve this approach with context descriptions and page hinting.

R to @sensiblehq: 2/ Document question answering at scale requires robust prompts and chunking strategies that withstand variability between documents. It's a notable challenge compared to plugins that allow you to chat with a single PDF. Here's a peek under the hood: 👇

1/ At @Sensiblehq, we've been leveraging LLMs to transform unstructured documents into structured data at scale. Here's everything you need to know about document question-answering, including: 👇 ✅ chunking ✅ layout preservation ✅ cost optimization ✅ confidence scores

R to @sensiblehq: Hey, Sensible is very good, are you trying to expand the brand ?