Granular Retrieval & Verification

The topic search (along with our other semantic apis) produce high level insights. In order to ensure these insights are trustworthy and actionable, we provide a mechanism to uncover the data that goes into the insights with our query API. This api shares a unified filtering engine with our higher level semantic apis: as a result, any semantic rollup or insight aggregation can be instantly “unrolled”.

Among other parameters, the query api support filtering on topic_id. As a result, we can pick any topic As a result, any aggregate statistic can be immediately converted into the excerpt that comprise the statistic.

Example: Generative AI in Search

Let’s take the topic Generative AI in Search. We can uncover the topic metadata below and see that it was mentioned 137 times in the corpus.

TOPIC_ID = 5
row = topic_df.loc[topic_df.topic_id == TOPIC_ID]
row[["topic_id", "short_title", "mentions"]]
topic_id short_title mentions
49 5 Generative AI in Search 137.0

Query: Generative AI in Search

We can call the index.query API, passing in our topic_id as well as a set of parameters to ensure we do not cut off our results prematurely. We can see that we have 137 mentions returned, lining up exactly with our aggregate apis. Below we display the first and last result of our search, as well as highlight a few terms to make the excerpts easier to read.

You will notice that accompanying each excerpt is a set of tags. These are the same tags that are returned in our topicSearch api. Here each tag corresponds to the percentage of the paragraph that it comprises.

df = index.query(topic_id=TOPIC_ID, max_excerpts_per_doc=20, limit=200) ## 200 is the single request limit
displayText(df.iloc[[0, -1]], highlight=["ai", "conversational", "search", "model", "multi"])
assert len(df) == row.iloc[0].mentions

Result 1/137

MSFT 2023Q4
  1. Comprehensive Security Solutions: 50%
  2. Microsoft 365 Innovations: 29%
  3. AI-Driven Developer Productivity: 14%
  4. Generative AI in Search: 4%

We rolled out the biggest update to Windows 11 ever adding 150 new features including new AI-powered experiences in apps, like, Clipchamp, Paint, and Photos, and we introduced Copilot in Windows, the Everyday AI companion, which incorporates the context to the web, your work data and what you are doing on the PC to provide better assistance. We are seeing accelerated Windows 11 deployments worldwide from companies like BP, Eurowings, Kantar, and RBC. Finally, with Windows 365 Boot and Switch, we’re making it easier than ever for employees at companies like Crocs, Hamburg Commercial Bank, the ING Bank to get a personalized Windows 365 Cloud PC with Copilot on any device. Now on to security. The speed, scale, and sophistication of cyberattacks today is unparalleled and security is the number one priority for CIOs worldwide. We see high-demand for Security Copilot, the industry’s first and most advanced generative AI product, which is now seamlessly integrated with Microsoft 365 Defender. Dozens of organizations including Bridgewater, Fidelity National Financial, and Government of Alberta have been using Copilot in Preview, and early feedback has been positive. And we look forward to bringing Copilot to hundreds of organizations in the coming months as part of the new early access program, so they can improve the productivity of their own, security operation centers and stop threats at machine speeds.

…

Result 137/137

META 2024Q2
  1. Zuckerberg on Business Strategies: 61%
  2. Generative AI in Search: 28%
  3. Smart Eyewear Integration: 5%
  4. AI Interaction in Business: 2%

Mark Zuckerberg: All right. So yes, on the Google and Microsoft partnerships, yes, I mean we work with them to have real-time information in Meta AI. It’s useful. I think it’s pretty different from search. We’re not working on search ads or anything like that. I think this will end up being a pretty different business.