Connect with us

AI Research

Office AI Science Team – Microsoft Research

Published

1 month ago

on


The Office AI Science team is part of OPG. The team builds systems that are leveraged across M365 and especially within Word, Excel, and PowerPoint. The team’s recent projects have included: PPT Summarization, Audio Overviews (Podcast), SPOCK Eval, Data Pipeline, Natural Language to Office JS, and CUA. 

PPT Summarization: The Office AI Science team built the first fine-tuned SLM within M365. The fine-tuned Phi-3 Vision SLM improved p95 latency of PPT Visual Summary feature from 13 seconds to 2 seconds, while maintaining quality (opens in new tab) on par with GPT-4o-v. The optimization resulted in 75 times fewer GPUs being used compared to GPT-4o-v and almost 9 times the number of PowerPoint users receiving a visual summary. The fine-tuned SLM also powers PPT Visual Q&A, making it both faster and cheaper. The team also introduced PPT Interactive Summary, which allows users to drill into visual summaries in more detail, leading to over 50% decline in thumbs down per 100k tries over 3 months, 30% interactivity clicking on chevron to go deeper, and a 17.6% increase in weekly return rate. The team is currently fine-tuning 4o-mini-vision with the goal of replacing remaining non-English traffic to GPT-4o-v with this smaller model and evaluating Phi-4 Vision for English. 

Audio Overviews: The team is building the Audio Overview Skill that introduces a podcast-like experience for consuming documents and artifacts. The feature is currently in the dogfood phase for MSIT, with production rollout scheduled for May 7 onwards. Users will be able to generate Audio Overviews from App Chat entry points in Word Win32 & Web, Copilot Notebooks (including OneNote), and other apps like Outlook Web, OneDrive Web and ODSP Mobile. Latest human evaluation (opens in new tab) scores overall transcript quality for the single file audio overview at 4.08/5.00 compared to 3.76/5.00 for NotebookLM, and with automated evaluation (opens in new tab), the team improved the overall score from an initial 4.09 to 4.65 with a two-step design leveraging GPT-4o and o3-mini. More details, including evaluation against multiple files for the Copilot Notebooks scenario and gains from moving to GPT-4.1, can be found here (opens in new tab). 

SPOCK (AugLoop Eval): In collaboration with AugLoop, the Office AI Science team developed several key features that enable agility in evaluating App Copilot scenario quality metrics. By the end of FY25Q3, 22 scenarios have been onboarded across Word, PPT, Office AI, and SharePoint, with Excel onboarding in-progress. The platform currently reliably runs 300 eval jobs and 30,000 tests daily. The automated scenario evaluation turnaround time compared to manual run has significantly decreased from days to 2-4 hours. SPOCK now supports intent detection, Leo Metrics, BizChat 1K Query, Python, and Typescript customer evaluators; model swap and FlexV3 eval are coming in Q4. Additionally, the v-team is automating the App Copilot Quality Dashboard (ÆVAL – Copilot Evaluation (opens in new tab)), providing a comprehensive overview of the quality of App Copilot scenarios. 

Data Pipeline: The team also created an online, self-serve, on-demand ADF pipeline for mining Office documents from the internet. This allows partners to kick off large-scale data mining jobs for specific languages and document types and features custom metadata extractors for extracting task-dependent document representations. By leveraging Bing’s precrawled 40B URL RetroIndex, document discovery is fast and efficient. OAI Science and several partner teams (Word+Editor, PPT Science, Word Designer, Designer, MSAI) are already utilizing the data for finetuning and test set creation. 

Natural Language to Office JS: The Office AI Science team is working to finetune o* family model for common Office scenarios like inserting slides from another PowerPoint file, inserting headers and footers in Word, or creating and finding merged ranges in Excel. 

CUA: The team also recently embarked on an exploration of Computer User Agent (CUA) centered on understanding user intent and adapting in real time. Leveraging plan assistance with the Office knowledge base, the team approximately doubled the task completion rate against OSWorld PPT scenarios. The team is working on fine-tuning the CUA model to improve task completions for Office apps. 

For more contact: Amanda Gunnemo or Vishal Chowdhary 





Source link

Related Topics:
Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

AI Research

Now Artificial Intelligence (AI) for smarter prison surveillance in West Bengal – The CSR Journal

Published

4 hours ago

on

September 7, 2025

By



Now Artificial Intelligence (AI) for smarter prison surveillance in West Bengal  The CSR Journal



Source link

Continue Reading

AI Research

OpenAI business to burn $115 billion through 2029 The Information

Published

7 hours ago

on

September 6, 2025

By


OpenAI CEO Sam Altman walks on the day of a meeting of the White House Task Force on Artificial Intelligence (AI) Education in the East Room at the White House in Washington, D.C., U.S., September 4, 2025.

Brian Snyder | Reuters

OpenAI has sharply raised its projected cash burn through 2029 to $115 billion as it ramps up spending to power the artificial intelligence behind its popular ChatGPT chatbot, The Information reported on Friday.

The new forecast is $80 billion higher than the company previously expected, the news outlet said, without citing a source for the report.

OpenAI, which has become one of the world’s biggest renters of cloud servers, projects it will burn more than $8 billion this year, some $1.5 billion higher than its projection from earlier this year, the report said.

The company did not immediately respond to Reuters request for comment.

To control its soaring costs, OpenAI will seek to develop its own data center server chips and facilities to power its technology, The Information said.

OpenAI is set to produce its first artificial intelligence chip next year in partnership with U.S. semiconductor giant Broadcom, the Financial Times reported on Thursday, saying OpenAI plans to use the chip internally rather than make it available to customers.

The company deepened its tie-up with Oracle in July with a planned 4.5-gigawatts of data center capacity, building on its Stargate initiative, a project of up to $500 billion and 10 gigawatts that includes Japanese technology investor SoftBank. OpenAI has also added Alphabet’s Google Cloud among its suppliers for computing capacity.

The company’s cash burn will more than double to over $17 billion next year, $10 billion higher than OpenAI’s earlier projection, with a burn of $35 billion in 2027 and $45 billion in 2028, The Information said.

Read the complete report by The Information here.



Source link

Continue Reading

AI Research

Who is Shawn Shen? The Cambridge alumnus and ex-Meta scientist offering $2M to poach AI researchers

Published

12 hours ago

on

September 6, 2025

By


Shawn Shen, co-founder and Chief Executive Officer of the artificial intelligence (AI) startup Memories.ai, has made headlines for offering compensation packages worth up to $2 million to attract researchers from top technology companies. In a recent interview with Business Insider, Shen explained that many scientists are leaving Meta, the parent company of Facebook, due to constant reorganisations and shifting priorities.“Meta is constantly doing reorganizations. Your manager and your goals can change every few months. For some researchers, it can be really frustrating and feel like a waste of time,” Shen told Business Insider, adding that this is a key reason why researchers are seeking roles at startups. He also cited Meta Chief Executive Officer Mark Zuckerberg’s philosophy that “the biggest risk is not taking any risks” as a motivation for his own move into entrepreneurship.With Memories.ai, a company developing AI capable of understanding and remembering visual data, Shen is aiming to build a niche team of elite researchers. His company has already recruited Chi-Hao Wu, a former Meta research scientist, as Chief AI Officer, and is in talks with other researchers from Meta’s Superintelligence Lab as well as Google DeepMind.

From full scholarships to Cambridge classrooms

Shen’s academic journey is rooted in engineering, supported consistently by merit-based scholarships. He studied at Dulwich College from 2013 to 2016 on a full scholarship, completing his A-Level qualifications.He then pursued higher education at the University of Cambridge, where he was awarded full scholarships throughout. Shen earned a Bachelor of Arts (BA) in Engineering (2016–2019), followed by a Master of Engineering (MEng) at Trinity College (2019–2020). He later continued at Cambridge as a Meta PhD Fellow, completing his Doctor of Philosophy (PhD) in Engineering between 2020 and 2023.

Early career: Internships in finance and research

Alongside his academic pursuits, Shen gained early experience through internships and analyst roles in finance. He worked as a Quantitative Research Summer Analyst at Killik & Co in London (2017) and as an Investment Banking Summer Analyst at Morgan Stanley in Shanghai (2018).Shen also interned as a Research Scientist at the Computational and Biological Learning Lab at the University of Cambridge (2019), building the foundations for his transition into advanced AI research.

From Meta’s Reality Labs to academia

After completing his PhD, Shen joined Meta (Reality Labs Research) in Redmond, Washington, as a Research Scientist (2022–2024). His time at Meta exposed him to cutting-edge work in generative AI, but also to the frustrations of frequent corporate restructuring. This experience eventually drove him toward building his own company.In April 2024, Shen began his academic career as an Assistant Professor at the University of Bristol, before launching Memories.ai in October 2024.

Betting on talent with $2M offers

Explaining his company’s aggressive hiring packages, Shen told Business Insider: “It’s because of the talent war that was started by Mark Zuckerberg. I used to work at Meta, and I speak with my former colleagues often about this. When I heard about their compensation packages, I was shocked — it’s really in the tens of millions range. But it shows that in this age, AI researchers who make the best models and stand at the frontier of technology are really worth this amount of money.”Shen noted that Memories.ai is looking to recruit three to five researchers in the next six months, followed by up to ten more within a year. The company is prioritising individuals willing to take a mix of equity and cash, with Shen emphasising that these recruits would be treated as founding members rather than employees.By betting heavily on talent, Shen believes Memories.ai will be in a strong position to secure additional funding and establish itself in the competitive AI landscape.His bold $2 million offers may raise eyebrows, but they also underline a larger truth: in today’s technology race, the fiercest competition is not for customers or capital, it’s for talent.





Source link

Continue Reading

Trending