Papersby GlyphServiceNow research papers
A company publishing mostly on agents and evaluation, best known here for WorkArena.21 papers · 976 citations · latest Oct 2026
Apriel Voice 27B Preview: An Open Model for Speech, Reasoning, and Tool UseApriel Voice 27B is an open AI model that directly processes spoken speech to converse, reason, and use external tools.
ServiceNow37 upvotesRead →AgentJudgeBench: A Multi-Difficulty Benchmark for Evaluating LLM Judges on Agentic Tool-CallingAgentJudgeBench reveals that LLM judges face structural reliability limits on dependency-driven agentic tool-calling workflows, with alignment degrading by difficulty and ground-truth exposure yielding mixed effects.ServiceNow4 citations19 upvotesRead →StarHarness: Evolving Harnesses with Stratified Search for Enterprise EnvironmentsStarHarness evolves fixed-weight agent harnesses via stratified task pools and hidden selection to improve enterprise tool-use performance and cross-model transfer.ServiceNow3 citations41 upvotesRead →SynthDocBench: Controlled Benchmark for Long-Context Visual Document UnderstandingSynthDocBench is a synthetic long-context visual document benchmark that isolates failure modes in vision-language models, revealing sharp length degradation, positional sensitivity, and chart comprehension breakdowns.ServiceNow1 citations68 upvotesRead →EVA-Bench: A New End-to-end Framework for Evaluating Voice AgentsEVA-Bench presents a comprehensive evaluation framework for voice agents that simulates realistic conversations and measures performance across multiple voice-specific failure modes using novel accuracy and experience metrics.ServiceNow20 citations77 upvotesRead →Do Enterprise Systems Need Learned World Models? The Importance of Context to Infer DynamicsEnterprise discovery agents that read system configuration at runtime outperform traditional world models in configurable environments where dynamics change over time.ServiceNow2 citations65 upvotesRead →Apriel-Reasoner: RL Post-Training for General-Purpose and Efficient ReasoningApriel-Reasoner is a 15B-parameter language model trained with reproducible multi-domain reinforcement learning to improve reasoning efficiency and accuracy across diverse tasks while reducing inference costs.ServiceNow12 upvotesRead →Therefore I am. I ThinkReasoning models appear to encode action choices before beginning textual deliberation, as evidenced by early decision detection and activation steering effects.ServiceNow2 citations31 upvotesRead →Terminal Agents Suffice for Enterprise AutomationSimple terminal-based coding agents using programmatic interfaces and foundation models can effectively perform enterprise tasks comparable to or better than complex tool-augmented agents.ServiceNow6 citations98 upvotesRead →CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use AgentsCUA-Suite introduces a large-scale ecosystem of expert video demonstrations and annotations for computer-use agents, providing continuous screen recordings and detailed reasoning annotations to advance desktop automation capabilities.ServiceNow8 citations97 upvotesRead →EnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise SettingsA sandbox of 164 database tables and 512 tools with 1,150 enterprise tasks; the best model tested completed only 37% of them.ServiceNow18 citations150 upvotesRead →Privileged Information Distillation for Language ModelsTraining methods that utilize privileged information for language model distillation in multi-turn environments outperform standard supervised fine-tuning followed by reinforcement learning approaches.ServiceNow105 citations29 upvotesRead →Grounding Computer Use Agents on Human DemonstrationsGroundCUA, a large-scale desktop grounding dataset, enables the development of GroundNext models that achieve state-of-the-art performance in mapping instructions to UI elements with less training data.ServiceNow23 citations107 upvotesRead →Apriel-1.5-15b-ThinkerA 15-billion parameter multimodal reasoning model achieves competitive performance through a progressive training methodology without reinforcement learning, demonstrating efficient use of computational resources.ServiceNow2 citations125 upvotesRead →DeepCodeSeek: Real-Time API Retrieval for Context-Aware Code GenerationA novel technique for predicting APIs and generating code in real-time using a compact reranker outperforms larger models with reduced latency, addressing API leaks and unclear usage intent in enterprise code.ServiceNow5 upvotesRead →Optimizing What Matters: AUC-Driven Learning for Robust Neural RetrievalA new training objective, MW loss, is introduced to improve retriever calibration and ranking quality by directly optimizing the Area under the ROC Curve (AUC), outperforming Contrastive Loss in retrieval-augmented generation tasks.ServiceNow4 upvotesRead →How to Train Your LLM Web Agent: A Statistical DiagnosisA study on compute allocation for post-training LLM-based web agents finds that combining supervised fine-tuning with on-policy reinforcement learning improves performance and reduces computational costs compared to using either method alone.ServiceNow23 citations52 upvotesRead →Multi-task retriever fine-tuning for domain-specific and efficient RAGInstruction-tuned retrieval encoder addresses domain-specific challenges for efficient, scalable, and fast Retrieval-Augmented Generation (RAG) applications.ServiceNow2 citations10 upvotesRead →The BrowserGym Ecosystem for Web Agent ResearchBrowserGym provides a standardized environment for evaluating web agents using LLMs, facilitating consistent benchmarking and improving agent development and analysis.ServiceNow200 citations24 upvotesRead →WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work TasksWorkArena++ benchmark evaluates the effectiveness of LLMs and VLMs in workplace tasks, revealing challenges and providing a mechanism for model fine-tuning.ServiceNow114 citations2 upvotesRead →WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?WorkArena and BrowserGym evaluate large language model-based agents' ability to perform enterprise software tasks, revealing gaps in current agent capabilities and differences between open and closed-source LLMs.ServiceNow443 citations2 upvotesRead →