By Dan Walsh

A new generation of Duncannon residents have grown up since 2007 with the absence of a Blue Flag, however, the tide has turned in 2026 and the community was excited by the Duncannon Blue Flag Raising Ceremony that took place on Friday.

Cathaoirleach of Wexford County Council, Cllr Joe Sullivan said; “It gives me great pleasure to be here today nestled under Duncannon Fort on the Ring of the Hook to help raise the Blue Flag at this magnificent beach after nearly two decades.

“The people of Duncannon have shown great determination and tenacity with the assistance of Wexford County Council to regain the Blue Flag.

“The achievement of Blue Flag status for Duncannon Beach coincides with the re-opening of Duncannon Fort and it is hoped that this will help bring further visitors to the area in the next few years.

“I am delighted to be present to witness the raising of the Blue Flag above Duncannon Beach once more. It is great that Duncannon takes its place with Ballinesker, Carne, Curracloe, Morriscastle and Rosslare Strand amongst the best beaches in the country,” concluded Cllr Sullivan.

Cllr John Dwyer, Cathaoirleach New Ross Municipal District said; “It was an absolute pleasure to be invited to speak at the raising of the Blue Flag for Duncannon beach today. All across the New Ross District, I have been quietly fighting the corner of our rural communities, and I will continue to raise issues that are brought to me.”

Ceann Comhairle and local Oireachtas member Verona Murphy said it was “a great day for the community of Duncannon raising its regained Blue Flag and a packed Community Centre celebrated the occasion and the rain didn’t dampen spirits.

“A massive achievement after nearly 20 years, they never gave up and helped by 37 farmers achieved their aim. Well done,” added Deputy Murphy.

A number of local groups were acknowledged for their contribution to regaining the Blue Flag including Duncannon Tidy Towns, Duncannon Village Renewal, Hook Tourism, local businesses, local community volunteers, Uisce Éireann, Department of Agriculture, Food and the Marine and Wexford County Council staff.

The attendance included Cllrs Marty Murphy, John Fleming and Michael Sheehan, Alan Fitzhenry, New Ross District Manager, Eamonn Hore, Deputy Chief Executive Wexford County Council, and other invited guests.

Eoin Kinsella from Wexford County Council Environmental Section acted as master of ceremonies.

Subscribe to the Frontier Red Team newsletter Get updates on our latest red-teaming research and findings. Michael Ilie, C. Daniel Freeman, and Kevin K. Troy In August 2025, we ran an experiment to see how much Claude could help Anthropic employees—who were not robotics experts—perform sophisticated (and amusing) tasks with an off-the-shelf robotic quadruped (henceforth, a robodog). We called this Project Fetch. We found that access to our state-of-the-art model at the time (Claude Opus 4.1) helped one team substantially outperform the other, who had to rely only on the internet and their own ingenuity. The Claude-enabled team got more done, faster. Before we dragged our colleagues to a warehouse for the experiment, we double checked whether Opus 4.1 could do the tasks entirely on its own. Unquestionably, it could not. Much like our team without Claude, it got hung up on the preliminary task of figuring out how to connect to the robot. But AI models are moving fast—even faster than the runaway robodog that almost rammed into one of our human teams back in August. We figured it was time to revisit Project Fetch to see if our newer models could outperform the previous generation. Not only did they do that, but Claude Opus 4.7—operating without human assistance—was about 20 times faster than the fastest human team at all tasks completed by our participants less than a year ago. This doesn’t mean that LLMs have now solved robotics. Far from it. The latest Claude models still struggled with using the robot to precisely move the beach ball—the “fetching” part of Project Fetch. And none of the tasks in these experiments implicate the more challenging, low-level elements of robotic control, such as developing a specific actuation policy. However, once again, we are seeing a pattern whereby first, models are helpful to humans. Then, humans are helpful to models. Finally, models are largely able to do things themselves. We have seen this in cybersecurity and now the same dynamics are starting to take shape at the intersection of AI and the physical world. The original Project Fetch had teams of Anthropic employees (randomly assigned to work with or without Claude) do the following steps: operate the robodog using the manufacturer-provided controller, connect to the robodog’s video and lidar sensors, write and operate a program to manually control the robodog, develop a way to monitor the robodog’s path through space, write a program to detect the beach ball, and finally put it all together to autonomously retrieve the ball. For this autonomous update, we couldn’t ask Claude to use a physical controller, nor did we evaluate the time it took a researcher to use the Claude-programmed controller to retrieve the ball (though we did confirm that it worked as intended). On the remaining subset of tasks, we ran three trials of Opus 4.7 using adaptive thinking with effort set to maximum in Claude Code. We measured the elapsed time for each objective and qualitatively assessed the models’ success. The role of our researcher was limited to plugging a laptop running Claude Code into the robodog, entering the initial prompt, approving commands, and approving the model to go to the next task. Very simply: on every task that was completed by at least one human team in August, Opus 4.7 completed the same task at least ten times faster.1 If you consider the four tasks that were completed by both human teams, Opus 4.7 was, on average, more than 37 times faster than Team Claude-less and more than 18 times faster than Team Claude. The table compares the speed of the original teams (Team Claude and Team Claude-less) to Opus 4.7 on all of the tasks we tested as part of Phase Two. Whereas the humans struggled to choose between multiple different approaches to interface with the dog’s sensors, Opus 4.7 was able to quickly identify the best path. Much of the code it wrote was effective on the first try (which was not the case for Team Claude or Team Claude-less in the original experiment). Indeed, we can see evidence of Opus 4.7’s efficiency when we look at the volume of code it generated: it was as or more successful than both human teams while producing almost ten times less code than Team Claude. Opus 4.7 was not perfect. For example, it defaulted to using an outdated object detection algorithm. But even then, it was able to work around this and arrive at an effective solution. We observed little within-task variance (in absolute terms) on completion times for steps the model finished. (Though the aforementioned suboptimal algorithm selection is likely why one of the beach ball detection trials took substantially longer than the others.) Overall, for the tasks in this experiment within its capability envelope, Claude is now quite reliable. (See the next section for an analysis of what Claude is still unable to do.) It is worth underscoring (as we did in our previous post) that this progress is not the result of a concerted effort to improve the robotics capabilities of our models. These improvements, like so many others in the history of LLM development, have emerged from much more general scaling. When using their hands, and with some practice, our humans were able to pilot the robodogs to gently nudge a beach ball back to the home base (a patch of fake grass) where the robots started. This required the ability to quickly perceive if the ball had gone off course, how that error related to the previous command, where the ball was now, and then how to adjust future inputs to more precisely move the ball. This is a kind of closed loop at which people excel (at least after making some mistakes and learning from them). In our Phase Two experiments, Claude struggled to capture this subtlety. Like the humans who reached the phase of needing to write a program for autonomous beach ball retrieval, Claude was able to move the robot behind the ball and position it to knock the ball back to the starting point. But the efforts to do so were poorly controlled and (again, like our human participants) not successful. One of our researchers with more robotics experience than our Phase One volunteers successfully accomplished the task of programming autonomous fetching. With more time and additional scaffolding, we think it is very likely that current generations of Claude could do the same. What we will be watching for next, though, is the ability of the models to accomplish this final task with the same speed and reliability they displayed on the other elements of Project Fetch. Writing about Phase One, we emphasized how LLMs could provide uplift to non-expert humans needing to use robots. This is even more true now than before. Models now complete what was previously pair-programming work between humans and models much more quickly by themselves, which means that people can more quickly transition to controlling and using the robots. And for some tasks, a human in the loop controlling the robot may still outstrip the AI model with its (virtual) hand on the D-pad. What is interesting and different is that we now seem much closer to a world where models will be able to use off-the-shelf physical tools with relative ease—at least for limited purposes. This is similar to how AI models used existing software editing tools like string-replace when they made the transition to more agentic coding. We are plausibly entering the early era of physical agentic AI. More research is needed to understand models’ ability to make these physical tools more bespoke, whether by writing control policies tailored to particular tasks or by designing robotic systems. And there may be substantial barriers to this more generalized vision of physically capable and adaptable language models. But as we have seen, apparently large distances in model capability can be traversed quickly. Models building their own software tools might have seemed outlandish not long ago, but it is happening. It would be unwise to rule out the same trajectory in hardware. Updated Jun 18: Corrected the date of the first phase of Project Fetch. In cybersecurity, a large fraction of real-world harm comes from N-days: vulnerabilities that have already been publicly disclosed, but only patched on some devices. In this post, we evaluate how much large language models can accelerate and automate the process of developing N-day exploits. Read moreGet updates on our latest red-teaming research and findings.

By Dan Walsh at Enniscorthy Municipal District Council monthly meeting

Enniscorthy Farmers Market is happy with its Abbey Square location, especially with the parking on Abbey Quay, and while it might be under review it is going nowhere at present.

That was the view of Enniscorthy District Manager, Claire Lawless, following a suggestion from Cllr John O’Rourke that the market be moved to a town centre location. The matter was raised at the June meeting of Enniscorthy Municipal District.

Cllr O’Rourke suggested that the Farmers Market be moved from its present location at Abbey Square to the old Dunnes Car Park. He also felt that allotments produce be incorporated into the Farmers Market. “It would help rejuvenate the town centre,” said Cllr O’Rourke.

This is not the first time that efforts were made to move the Farmers Market into the town centre, however, the Abbey Square site has been a permanent fixture for many years, and stall holders find it satisfactory to trade in this popular open-air environment beside the River Slaney.

Enniscorthy Farmers Market takes place every Saturday from 9am to 2pm.

This market features a wide selection of fresh produce, artisan foods, and handmade goods, all sourced from local farmers and makers.

Whether you’re looking for seasonal vegetables, locally reared meats, homemade treats, unique crafts, or fresh fish from Kilmore Quay, there’s something for everyone.

By Dan Walsh

Kilmore Quay RNLI launched Monday afternoon to assist the crew of an 8-metre motorboat with engine failure adrift in fog near the Saltee Islands.

The volunteer crew were requested to launch their all-weather Tamar class lifeboat RNLB Killarney by the Irish Coast Guard at 2.43 pm and under Coxswain Aidan Bates and with five crew members on board promptly launched and made its way to the scene.

Visibility was poor with dense fog covering the area. Sea conditions at the time were described as slight.

Arriving at the location at 3.06 pm, the lifeboat crew checked that everyone on board the vessel was safe and well before assessing the situation.

Taking into consideration the fog and position of the boat, it was decided that the safest course of action was to establish a tow to bring the disabled craft back to shore.

The lifeboat was soon under way to the nearest available harbour at Kilmore Quay arriving at 3.56 pm.

Building Reliable Agentic AI Systems A Case Study in building production-ready agentic AI systems This paper presents the Preclinical Information Center (PRINCE), a cloud-hosted platform developed by Bayer AG with Thoughtworks to address pharmaceutical industry challenges in drug development. PRINCE leverages Agentic Retrieval-Augmented Generation and Text-to-SQL to integrate decades of safety study reports. We describe PRINCE's evolution from keyword-based search to an intelligent research assistant capable of answering complex questions and drafting regulatory documents. We reflect on key engineering decisions through the lens of context engineering—how information was shaped and routed between specialized agents—and harness engineering—how orchestration, recovery, and observability were built around the models to maintain control and reliability. The system prioritizes trust through transparency, explainability, and human-in-the-loop integration. PRINCE demonstrates AI's transformative potential in pharmaceuticals, significantly improving data accessibility and research efficiency while ensuring governance and compliance. 16 June 2026 Contents - The Challenge: Navigating the Preclinical Data Maze - The Solution: PRINCE - An Evolutionary Platform - System Architecture: Engineering a Reliable Agentic RAG System - The Agentic RAG System - Building Trust in a Production LLM System - Engineering for Resilience: Error Handling and Recovery - Enhancing Data Quality: Named Entity Recognition and Annotation - The Journey Continues: Iterative Development - Conclusion Preclinical drug discovery is inherently complex and data-intensive. Researchers face the significant challenge of efficiently accessing and analyzing vast volumes of information generated during this critical phase. Traditional keyword-based search methods, often reliant on rigid Boolean logic, frequently fall short when confronted with the nuanced and intricate nature of preclinical research questions. The advent of Large Language Models (LLMs) has presented a transformative opportunity. By combining the generative power of LLMs with the precision of information retrieval systems, Retrieval-Augmented Generation (RAG) has emerged as a promising technique. This approach holds the potential to revolutionize preclinical data access, enabling researchers to pose complex questions in natural language and receive accurate, context-rich answers grounded in proprietary data. Recognizing this potential early, Bayer committed to exploring how these technologies could address longstanding challenges in preclinical research. In this post, we share that journey—how Bayer's early investment in generative AI has resulted in PRINCE, an agentic AI system built on Agentic RAG. This case study explores the technical architecture, engineering decisions, and lessons learned in transforming preclinical data retrieval from a challenging maze into an intuitive conversational experience. Many of the engineering decisions behind PRINCE can now be understood through the lens of context engineering and harness engineering, although when the system was first designed we did not use these terms. Context engineering shaped what information each model received, what it did not receive, and how context moved between specialized steps such as research, reflection, and writing. Harness engineering shaped the scaffolding around the models: orchestration, tool boundaries, state persistence, retries, fallbacks, validation, reflection loops, observability, and human review. While this post focuses on the technical architecture and engineering challenges, our paper published in Frontiers in Artificial Intelligence covers the product evolution and business impact in more detail. The Solution: PRINCE - An Evolutionary Platform To address these challenges, Bayer developed the Preclinical Information Center (PRINCE) platform. PRINCE was conceived as a unified gateway to preclinical data, initially focusing on consolidating previously siloed structured study metadata and exposing them in a “Searchable” manner. This initial phase allowed users to apply advanced filters and retrieve information primarily from structured study metadata. However, a significant portion of Bayer's valuable preclinical knowledge resides within unstructured PDF study reports accumulated over decades. Due to numerous system migrations over the years, the structured metadata associated with these reports could be incomplete, missing, or even contain incorrect annotations. Crucially, the authoritative “gold standard” information was consistently present within the approved PDF study reports. The emergence of Generative AI, particularly RAG, provided the key to unlocking this wealth of unstructured data. By integrating RAG capabilities, PRINCE began to shift the paradigm from a filter-based 'search' tool to a natural language 'ask' system, enabling researchers to query the content of these study reports directly. This evolution reflects PRINCE's progression through three distinct phases: - Search: the initial phase focused on creating a unified gateway to thousands of nonclinical study reports, consolidating multiple in-house data silos from various preclinical domains into a searchable format, primarily leveraging structured metadata. - Ask: this phase introduced an AI-powered question-answering system utilizing Retrieval Augmented Generation (RAG). This enabled researchers to derive insights directly from unstructured data, including scanned PDFs from historical reports, by posing questions in natural language. - Do: the current phase positions PRINCE as an active research assistant capable of executing complex tasks. This is achieved through the integration of multi-agent systems, allowing the platform to handle intricate queries, orchestrate workflows, and support activities like drafting regulatory documents. This deliberate evolution from Search to Ask to Do represents a strategic response to the industry's need for greater efficiency and innovation in preclinical development. By providing researchers with increasingly powerful tools to access, analyze, and act upon preclinical data, PRINCE aims to enable faster data-driven decision-making, reduce the need for unnecessary experiments, and ultimately accelerate the development of safer, more effective therapies. System Architecture: Engineering a Reliable Agentic RAG System The system functions as an interactive conversational UI, powered by a robust backend infrastructure. Its architecture, designed for handling complex queries and delivering accurate, context-rich answers, is orchestrated using LangGraph and served via a FastAPI application. Figure 1 provides the system context—UI, backend, data stores, LLM fallbacks, and observability—while Figure 2 zooms into how the system coordinates its specialized agents. Figure 1: System context and supporting platforms. - User Request: the process begins when a user submits a request through the Conversational UI which is built with React. - Orchestration: the user's request is routed to a LangGraph-based orchestration layer in the backend. This workflow engine coordinates a multi-stage process that progresses through clarifying user intent, thinking and planning, conducting research (using RAG and Text-to-SQL), validating data completion, and finally generating a response through the Writer agent. The workflow includes deliberate pause points and feedback loops to ensure data completeness before proceeding. (We explore the details of this agentic workflow in a dedicated section later.) - Data Retrieval and State Management: the Researcher agents interact with a comprehensive and distributed data ecosystem: - Vector representations of all study reports are stored in OpenSearch, forming the core knowledge base for information retrieval. - Curated structured data, resulting from various ETL and harmonization processes, is accessed via Athena. - The state of the agent's execution is meticulously tracked. After each logical step (a LangGraph node execution), the corresponding state is persisted in PostgreSQL using a LangGraph checkpointer. - Broader application-level state is managed in DynamoDB. - The system leverages internal GenAI platforms that host models from OpenAI, Anthropic, Google, and open-source providers. These platforms expose all models via a unified OpenAI-compatible endpoint, making it easy to swap models and choose the best tool for each task. They also manage the control plane, enforcing rate limits and other safeguards to prevent abuse. - Resilience and Error Handling: robustness is a critical design principle, with multiple fallback mechanisms in place: - If a specific LLM fails, the system automatically retries the request several times before falling back to an alternative model or platform to ensure service continuity. - To recover quickly from transient failures, retries are implemented at both the individual LLM call level and the logical node level (i.e., an entire step in the agent's plan). - Also, agents are provided the context of the errors so that they can chart a different trajectory or alternative plan of action as a response. - Observability and Evaluation: the entire system is monitored for performance and reliability: - General system health and metrics are tracked using Cloudwatch. - Langfuse serves as the primary observability tool, providing detailed traces of all production traffic. This allows for in-depth debugging of issues. Furthermore, evaluation datasets are stored and managed within Langfuse, making it easier to analyze performance scores and diagnose specific failures. The evaluation is done using RAGAS evaluation framework. The live traffic evaluation is done on a daily basis while the dataset evaluation is done whenever significant changes are made to the core workflow, prompts, or underlying models. - Final Response: once the agents have processed the request and generated a satisfactory response, it is sent back to the Conversational UI to be presented to the user. A design principle running through this architecture is context discipline. Larger context windows did not remove the need to be selective about what each agent sees. In early iterations, putting too much information into the context made the system harder to steer and harder to evaluate. PRINCE therefore avoids treating the prompt as one large container for all available information. Instead, different stages receive different context: planning context for Think & Plan, retrieval context for the Researcher Agent, evidence context for the Reflection Agent, and synthesis context for the Writer Agent. This reduces context pollution and makes the system easier to debug, evaluate, and improve. These steps ensure that the system can provide reliable and contextually relevant answers to a wide range of complex queries by leveraging a sophisticated, multi-agent architecture and a diverse set of powerful tools and data sources. The Agentic RAG System PRINCE incorporates an agentic RAG system (Figure 2) to handle complex user requests that require multiple steps, reasoning, and interaction with different tools or data sources. This setup, implemented using LangGraph, orchestrates the overall workflow and leverages Researcher Agent, Writer Agent, and Reflection Agent for specific tasks. The system is designed to be robust and reliable, with multiple fallback mechanisms in place to ensure that the system can continue to function even if some of the components fail. Figure 2: The research workflow. Clarify User Intent The Clarify User Intent step serves as the first line of defense against ambiguity. As the system scaled to include diverse domains like toxicology and pharmacology, simple user queries often became ambiguous, making it difficult to automatically select the right tools. Rather than relying on expensive trial-and-error across all data sources, the system proactively asks clarifying questions to pinpoint the specific domain or data type. This ensures the system enhances the query with the necessary constraints to target the correct tools. We are also optimizing this by developing domain-level selection in the UI, which will allow users to pre-filter valid tools upfront. To further reduce friction, the system also provides AI-assisted source recommendations: when a user has not selected any data source — or has selected several without a clear focus — the model analyzes the intent behind the user's query and suggests the most relevant sources. The user retains full control and can accept, adjust, or override the recommendation, ensuring domain expertise always has the final say. This “fail-fast” mechanism prevents wasted execution on vague queries, while careful tuning ensures the system remains unobtrusive when the intent is already clear. From a context engineering perspective, this step is the first assembly decision in the workflow: it constrains which tools, domains, and data sources will be in scope before any retrieval begins, ensuring subsequent agents receive a focused rather than open-ended problem. Think & Plan: Process Reflection The Think & Plan step is responsible for devising a strategy to fulfill the user's request. This critical component gives the system a dedicated space to reason about the next steps before taking action—a technique inspired by Anthropic's Think tool. Importantly, this step performs process reflection: evaluating whether the agent is making the right progress toward its end goal and is on right trajectory, rather than evaluating the data itself. In multi-step agentic workflows, particularly those involving many sequential actions, process reflection is essential. Consider a scenario where the system needs to execute 50 steps to complete a complex task. At each juncture, the system must ask: Am I taking these steps in the right manner? Am I making the progress I'm supposed to make? Is the current trajectory leading toward the user's goal? The Think & Plan step provides this metacognitive capability, allowing the system to reflect on its own workflow and adjust its strategy accordingly. This “thinking space” has proven particularly valuable in scenarios involving multiple tool calls. When PRINCE was initially developed, it had only a couple of tools: one for RAG-based retrieval and another for Text-to-SQL queries. However, as we integrated more data sources to expand the system's capabilities, the number of available tools grew significantly. With this explosion of tools came an inherent challenge: overlapping concerns and domain boundaries across different tools. For example, multiple tools might serve similar but subtly different purposes—querying structured metadata versus unstructured reports, or retrieving study summaries versus detailed experimental data. When presented with tools that belong to similar domains but handle slightly different data, the LLM would sometimes struggle to select the most appropriate tool for a given query. By introducing a dedicated thinking step, the system can explicitly reason about which tool best matches the user's intent, evaluate the characteristics of each available tool, and make a more informed decision. This approach led to a dramatic improvement in the accuracy of tool selection. Beyond tool selection, the Think & Plan step is essential for orchestrating multi-step processes. Many complex queries in PRINCE require a series of tool calls where the output of one tool must be analyzed before determining the next action. For instance, the system might first query structured metadata to identify relevant studies, then use those study IDs to retrieve detailed information from unstructured reports, and finally synthesize the findings. Without a dedicated space for process reflection, the system would attempt to execute these steps linearly without evaluating whether each step is bringing it closer to the goal. With the thinking step in place, the system can pause, assess its progress in the workflow, and intelligently plan the subsequent tool calls needed to complete the user's request. The Researcher Agent The Researcher Agent serves as the system's primary information gatherer. As we onboard new scientific domains onto PRINCE, we consistently observe that data falls into two primary categories: structured and unstructured. While specific implementation techniques may vary across domains — for instance, leveraging Snowflake Cortex Analyst for pharmacology queries for Text-to-SQL versus other more custom methods for toxicology—the fundamentals behind these retrieval strategies remain consistent. As PRINCE expands across multiple preclinical domains, a single Researcher agent with a flat tool list becomes increasingly hard to manage. Many tools operate on similar concepts—“studies”, “findings”, “assays”—but point to different underlying datasets, schemas, and regulatory interpretations depending on the domain. For example, when a user refers to “the study”, the relevant context might be a repeat‑dose toxicology study, a cardiovascular safety pharmacology package, or a particular assay in aggregated mass‑data tables, each with its own preferred sources of truth. To avoid one monolithic agent juggling overlapping tools and subtly different data contracts, we are actively evolving the Researcher capability into a hierarchy of domain‑specific sub‑agents. In this proposed architecture, each domain agent will own its own toolset (for example, toxicology RAG + tox metadata SQL, or pharmacology RAG + assay‑level SQL) along with tailored prompt instructions that encode how that domain’s data model works, which tables or indices are authoritative, and how to interpret key concepts. We anticipate this will keep responsibilities coherent, reduce accidental cross‑domain leakage, and make it easier to reason about and test retrieval behaviour per domain. To effectively harvest insights from this diverse landscape, the Researcher Agent employs a hybrid retriever approach focused on two distinct patterns: - Retrieval-Augmented Generation (RAG): for processing unstructured data, primarily PDF reports. - Text-to-SQL: for querying structured data housed in Amazon Athena. This dual-strategy allows the system to bridge the gap between narrative scientific reports and quantitative experimental data. In this updated vision, the top‑level Researcher Agent is designed to act as a coordinator rather than a single all‑knowing component. Given the clarified user intent and any explicit domain selection from the UI, it will route the query to the appropriate domain sub‑agent, which can then decide how to combine RAG and Text‑to‑SQL within its own boundary. This pattern aims to preserve the simplicity of “one researcher” from the user’s perspective, while internally allowing each domain to evolve its own tools, schemas, and retrieval recipes without destabilizing the rest of the system. Retrieval-Augmented Generation (RAG) for Unstructured Data Given the vast repository of thousands of preclinical study reports and other unstructured documents, RAG is essential for extracting relevant insights by grounding LLM responses in this specific knowledge base. The RAG pipeline comprises a comprehensive ingestion process and a sophisticated query-time architecture. Ingestion Process: Preclinical study reports, mostly PDFs spanning decades and often including scanned documents with complex tables, are first centralized into an S3 data lake and passed through an extraction pipeline tuned for this corpus. The extracted text is normalized into structured JSON and then chunked using a strategy that preserves enough scientific context while keeping chunks efficient for retrieval. Each chunk is enriched with study‑ and section‑level metadata from Amazon Athena (for example study ID, compound, species, route, page, and parent section), which later enables precise metadata filtering in the RAG layer. Finally, these annotated chunks are embedded and indexed in Amazon OpenSearch Service, forming the vector store that backs semantic and metadata‑aware retrieval over both the historical corpus and the daily deltas as new or updated reports arrive. Query-Time RAG Pipeline: When a user submits a query, the system initiates a multi-stage retrieval process. This pipeline is engineered to effectively retrieve the most relevant and trustworthy information from the vector database to ground the LLM's response. To illustrate this pipeline, consider the example query: “Were any of the following clinical findings observed in study T123456-2: piloerection, ataxia, eyes partially closed, and loose faeces?”. The system processes this query through the following steps: - Keyword Extraction: the user's natural language query is first analyzed by an LLM. Through careful prompt engineering, the model is instructed to extract keywords highly relevant for keyword search within our document corpus (e.g., “piloerection”, “ataxia”, “eyes partially closed”, “loose faeces”). - Metadata Filter Generation: concurrently, the LLM generates a metadata filter based on the query. For example, a filter eq(study_id, T123456-2) is extracted to narrow the search space. This filter is dynamically generated using few-shot prompting with various permutation and combination examples provided to the model, ensuring it can handle diverse filtering requests. - Query Expansion: to ensure comprehensive retrieval and account for variations in phrasing and terminology, query expansion (multi query or query rewrite) is performed by a smaller, faster model. This generates n=5 semantically similar queries based on the original question. For the example query, this might include variations like: - “Clinical symptoms reported in research T123456-2, including goosebumps, lack of coordination, semi-closed eyelids, or diarrhea.” - “Recorded observations in experiment T123456-2 regarding hair standing on end, unsteady movement, eyes not fully open, or watery stools.” - “What were the clinical observations noted in trial T123456-2, particularly regarding the presence of hair bristling, impaired balance, partially shut eyes, or soft bowel movements.” - Hybrid Retriever: information retrieval from the vector database (Amazon OpenSearch Service) utilizes a Hybrid Search approach that combines metadata filtering, semantic vector similarity search (kNN), and keyword-based retrieval. This process is executed as follows: - Metadata Filtering: the metadata filter generated in the previous step (e.g., eq(study_id, T123456-2)) is applied directly to the vector database query. This pre-filters the search space based on the structured metadata attached to the chunks during the ingestion process from Amazon Athena, ensuring that only chunks associated with the specified study ID (or other relevant metadata) are considered. This significantly reduces the search space from millions of vectors to a more manageable range of tens to hundreds, improving efficiency and relevance. - Parallel Hybrid Search Execution: for each of the n=5 expanded queries, a single hybrid search query is executed in parallel against the filtered Amazon OpenSearch Service vector database. This query combines both semantic vector similarity search (kNN) and keyword-based search, leveraging OpenSearch's capabilities for efficient multi-vector and text search. - Weighted Result Scoring: within each individual hybrid search executed in parallel, a weighted approach is applied to the results. A weight of 0.7 is given to the semantic search results and 0.3 to the keyword search results to balance contextual understanding and precise term matching. This weighting was determined through experimentation to optimize retrieval effectiveness for our data. - Result Aggregation and Initial Ranking: the results (sets of relevant chunks with their weighted scores) from all 5 parallel hybrid search executions are aggregated. Unique chunks from all search results are pulled together, and their highest weighted score across the parallel searches is used to determine an initial ranking. This step initially retrieves a larger set of potential context chunks (k=~20) based on these aggregated and weighted scores. - Reranking: the initial set of retrieved chunks (k=~20) is then refined using a Rerank step. A cross-encoder model (bge-reranker-large) evaluates the relevance of each retrieved chunk against the original question, selecting the top k=7 most relevant chunks to be used as context for the LLM. This reranking step is crucial for ensuring that the most pertinent information, even if not the highest in initial semantic similarity or keyword match, is prioritized for the final response generation. - Final LLM Prompt Generation: the refined context (k=7 chunks) is then combined with the original question to form the final LLM prompt. This prompt is carefully constructed to guide the LLM in generating a focused and accurate response based on the provided context, minimizing the risk of hallucination. - Response Generation with Citation: a state-of-the-art reasoning model then processes the final prompt and the provided context to generate response with citation. The LLM synthesizes the information from the context to formulate a coherent and accurate answer. Crucially, the response automatically includes citations linking back to the specific chunks in the original document(s) that support the generated answer. - Monitoring: the entire Query-Time RAG process, from initial query to final response generation, is continuously monitored using Langfuse for observability, performance and quality analysis. Text-to-SQL for Structured Data While RAG excels at unstructured data, queries requiring precise filtering, aggregation, or comparison of structured data points are better suited for Text-to-SQL. Examples include “Give me 50 example studies done on RAT” or retrieving specific numerical assay results including dosage groups. As shown in the Researcher Agent can intelligently decide to hand over such queries to the Text-to-SQL tool. Figure 3: Text-to-SQL tool The process for converting a natural language question into an executable SQL query and retrieving results involves several key steps: - Query Analysis and Intent Recognition: the user's natural language query is analyzed to understand the user's intent and identify the specific data points and filters being requested from the structured metadata. - Schema Understanding and Relevant Schema Selection: to accurately generate a SQL query, the LLM requires an understanding of the relevant database schema. For large and complex schemas, only the necessary schema components relevant to the user's query are dynamically injected into the LLM's context. This reduces the complexity for the model and improves the accuracy of the generated SQL. - Dynamic Few-Shot Prompting for SQL Generation: converting complex natural language queries into precise SQL dialect (in our case, Athena) can be challenging for LLMs. To address this, we employ dynamic few-shot prompting. A collection of carefully hand-picked examples, representing various complex query patterns and their corresponding correct SQL translations in the Athena dialect, is stored in a separate collection within our vector database. Based on the user's query, relevant examples are retrieved from this “semantic layer” using vector similarity search and included in the prompt to the LLM. This provides the LLM with in-context learning examples, guiding it to generate accurate SQL queries in the correct dialect. Continuous addition of new examples based on encountered challenges further improves the system's performance over time. - SQL Query Generation and Validation: a model with strong code generation capabilities, conditioned on the relevant schema information and dynamic few-shot examples, generates the corresponding SQL query. To ensure the LLM can accurately process the results and identify the correct rows for subsequent synthesis, certain essential columns, such as study ID and study title, are always included in the generated SELECT query. The generated query is then validated to ensure it adheres to allowed operations (e.g., only SELECT queries are permitted; DELETE, INSERT, or UPDATE queries are explicitly blocked for data integrity and security). Notably, an earlier iteration of this process included an LLM review step for generated SQL queries; however, this step was later removed as it was found that the reviewing LLM sometimes incorrectly flagged valid queries as erroneous, hindering efficiency without a commensurate gain in accuracy. - Query Execution and Result Limiting: the validated SQL query is executed against the structured metadata database in Amazon Athena. To prevent data flooding and manage response size, the system enforces a limit, fetching not more than 50 records at a time. - Error Handling and Iteration: if the SQL query execution is successful, the retrieved results (up to the specified limit) are returned and integrated into the overall response generation process. If the query fails due to syntax errors, schema issues, or other execution errors, the error message from the database, along with the generated query and the original context, is passed back to the same model. The LLM analyzes the error and the context to generate a corrected SQL query. This iterative process of generating and executing SQL queries is attempted up to 3 times before the tool gives up and reports a failure, potentially indicating an unresolvable query or a limitation in the model's ability to handle the specific request. The Reflection Agent: Data Validation and Sufficiency While the Think & Plan step provides process reflection, the Reflection Agent performs a complementary but distinct type of reflection: data reflection. This crucial component evaluates whether the data retrieved from various tools is sufficient and relevant to answer the user's question—a fundamentally different concern from whether the workflow itself is progressing correctly. In multi-step agentic workflows, these two types of reflection serve different but equally important purposes. Process reflection (Think & Plan) ensures the agent is taking the right steps and making appropriate progress toward the goal. Data reflection (Reflection Agent) ensures that the information gathered through those steps is adequate to fulfill the user's request. Both are essential: an agent might execute a perfectly valid workflow (good process) but still retrieve insufficient data to answer the question, or conversely, might have access to sufficient data but fail to progress effectively through the workflow. As illustrated in the research workflow diagram (Figure 2), after initial information retrieval and 'think & plan' loops, the Reflection Agent is invoked when Think & Plan step thinks that the process has progressed well enough and is ready to evaluate the data. 'Reflection Agent' evaluates the sufficiency and relevance of the collected data by comparing the retrieved context against the user's original query and identifying potential gaps or missing information. If the gathered information is deemed insufficient to provide a complete response, the Reflection Agent generates specific follow-up questions designed to acquire the necessary missing information. These follow-up questions are then handed back to the Think & Plan step, which initiates further retrieval steps to obtain more comprehensive results. This iterative process of data validation and subsequent information retrieval, driven by the Reflection Agent's generated questions, demonstrates the system's ability to refine its search strategy based on the initial results. If the information is sufficient, the workflow proceeds to the next step. The Writer Agent: Answer Synthesis and Formatting Once the Researcher Agent has collected the relevant evidence from RAG and Text-to-SQL, the Writer Agent is responsible for turning that raw material into the final answer shown to the user. Its job is not to “discover” new information, but to synthesize the retrieved context, respect user instructions, and enforce PRINCE's quality constraints during generation. The Writer Agent operates with a few non-negotiable rules. It must ground every claim in the supplied context and attach accurate citations back to the underlying chunks and study IDs, since verifiability is critical in a regulated environment. It is also responsible for honoring user-level formatting requirements (for example, tables, bullet points, or specific section structures) and for aligning with domain-specific answer standards used by the preclinical scientists. For more complex responses—such as multi-section summaries or partially filled regulatory templates—the architecture supports extending the Writer Agent with a short internal review loop. In this pattern, the Writer would first draft an answer, then a reviewing step would check for missing sections, inconsistent tables, or gaps relative to the original question, and may send targeted instructions back to the Writer to revise specific parts. This design enables a lightweight form of reflection focused on answer completeness and presentation, complementing the Reflection Agent's focus on data sufficiency earlier in the workflow. Importantly, all outputs from these regulatory drafting workflows are intended for expert review; final submissions are authored and approved by qualified personnel. This gives PRINCE three complementary reflection loops. Process reflection checks whether the workflow is on the right path and helps catch bad trajectory, wrong tool choice, or poor sequencing. Data reflection checks whether the gathered evidence is sufficient and helps catch thin evidence, missing context, or gaps in coverage. Draft reflection checks whether the generated output is complete and helps catch missing sections, incomplete tables, or synthesis gaps. Together, these agents form a practical context engineering pattern. The system does not simply keep adding more information to the prompt. It routes the right context to the right capability at the right time: planning context for Think & Plan, retrieval context for the Researcher, evidence context for the Reflection Agent, and synthesis context for the Writer. This plays out in concrete decisions throughout the system: the Text-to-SQL step injects only the schema components relevant to the current query rather than the full database schema; the Reflection Agent receives the original question alongside collected evidence to assess gaps, not the full workflow history; and the Writer Agent receives curated chunks with citation constraints, not raw retrieval output. Moving from a monolithic agent to this structured workflow meant each agent could be evaluated, debugged, and improved in isolation. Building Trust in a Production LLM System Building and maintaining user trust is paramount for the successful adoption of any AI system, particularly in a critical environment like preclinical drug discovery where decisions have significant implications. For a production LLM application, trust is not just about accuracy; it's also about reliability, transparency, and the ability for users to verify the information provided. Several mechanisms are integrated into PRINCE to achieve this: Transparency and Explainability Ensuring transparency and explainability is a critical aspect of PRINCE's design, fostering user trust and enabling verification of the generated responses. The system incorporates several mechanisms to achieve this: - Intermediate Steps and Transparency: given the iterative nature of the workflow and the potential time required to generate a final answer, maintaining transparency is crucial. The intermediate steps executed by the system during query processing, information retrieval, and reflection, including the queries formulated and the tools utilized, are displayed to the user. This provides visibility into the system's reasoning process and allows users to follow the steps taken to arrive at the final answer. Additionally, when relevant context (chunks) is identified, links to these source materials are presented on the screen, allowing users to see precisely which information was shortlisted and used to formulate the final response. - Factuality Verification through Citation: the system facilitates user verification of factuality through a robust citation mechanism. The generated answer is consistently accompanied by citations referencing the original source documents and structured metadata. These citations are directly linked to the context displayed to the user, enabling them to easily verify the accuracy of the claims made in the response and trace the information back to its origin. Users can hover over any sentence in the generated response to see the corresponding citation, which provides a link to the PRINCE and to the source document, including the page number and the exact quote from the report used to support that part of the answer. This granular level of citation significantly enhances the credibility and trustworthiness of the system's output and simplifies the human review process. Evaluation Rigorous evaluation is fundamental to building and maintaining a reliable LLM application. PRINCE's performance and reliability are assessed through a combination of two types of evaluations: Dataset Evaluations and Live Traffic Evaluations. - Dataset Evaluations: conducted whenever significant changes are made to the core workflow, prompts, or underlying models, these evaluations utilize curated datasets with pre-defined reference answers, meticulously prepared by subject matter experts and stored in Langfuse. A custom evaluation script processes each question and compares the generated response against the reference answer, yielding quantitative metrics such as Faithfulness (degree to which the answer is supported by context), Answer Relevancy (how well the answer addresses the query), Context Relevancy (relevance of retrieved chunks), Answer Accuracy (comparison to ground truth), and Semantic Similarity with Reference (semantic similarity to reference answer). Given the agentic nature of the system, applying appropriate evaluation metrics at different workflow stages, analogous to a testing pyramid, is crucial in addition to evaluating overall end-to-end performance. - Live Traffic Evaluations: performed daily as a batch job on real user queries from the live environment (without pre-defined reference answers), these evaluations provide valuable insights into real-world performance. Metrics such as Faithfulness and Answer Relevancy can still be assessed. Live traffic evaluations are essential for monitoring system behavior, identifying potential issues like hallucinations in production, and understanding performance on diverse live queries. Monitoring Continuous monitoring of the system's performance and outputs is essential for proactive identification and resolution of issues in a production environment. Using platforms like Langfuse, we continuously monitor PRINCE to identify potential biases, errors, or areas for improvement, ensuring the reliability and safety of the system's responses. Engineering for Resilience: Error Handling and Recovery Given the complexity of the multi-step workflow inherent in PRINCE, robust error handling and recovery mechanisms are critical to ensure the system's reliability and provide a seamless user experience. The system is engineered to recover gracefully from failures at various stages without requiring a complete restart of the entire workflow. Key aspects of our error handling and recovery approach include: - State Persistence: the state of the entire workflow graph is persistently stored, enabling the system to resume execution directly from the failed node. This is achieved by storing the Agent State, representing the progress of the agents through the workflow, in Postgres. Other aspects of the application state, such as logs, intermediate steps, and citations, are stored in DynamoDB. This separation and persistence of state are crucial for achieving robustness in a stateful agentic system. - Built-in Retries: the system is configured with built-in retries at various steps in the workflow. If a particular step encounters a transient failure, the system will automatically attempt to re-execute it a predefined number of times before signaling a more permanent error. - User-Initiated Retries: in addition to automated retries, users have the option to manually retry a failed query through the interface. When a user initiates a retry, the system leverages the persisted state to continue the workflow directly from the point of failure, intelligently skipping the steps that were successfully completed in the previous attempt. This significantly improves user experience and saves computational resources. - Framework-Level Support: the error recovery mechanisms are significantly supported by the underlying framework, LangGraph, which offers solid built-in capabilities for managing workflow state and handling errors within the graph structure. This provides a robust foundation for building resilient agentic workflows. - LLM Fallbacks: to enhance reliability and mitigate issues related to model availability or performance, the system incorporates custom LLM fallback handling. If a call to a primary LLM provider or a specific model fails after a few retries, the system automatically falls back to an alternative LLM from a different provider. This mechanism is crucial for maintaining system availability and responsiveness, especially as platform downtimes for external services are outside of our direct control. This comprehensive approach to error handling and recovery minimizes the impact of transient failures, reduces the need for users to restart complex queries from scratch, and contributes to cost and latency savings by avoiding redundant execution of successful steps and LLM calls, all of which are essential for a production-ready system. These mechanisms are harness engineering in practice. The LangGraph workflow acts as the control layer around the agents: it defines which component can act, which tools it can use, where the workflow can pause, how failures are retried, how state is persisted, and when the system should move from research to reflection to writing. This harness makes the system less opaque and more reliable than an unconstrained autonomous agent. It gives the application clear control points for recovery, inspection, evaluation, and human intervention. Enhancing Data Quality: Named Entity Recognition and Annotation The accuracy and completeness of the structured metadata in Amazon Athena are critical for the performance of the Text-to-SQL component and overall data discoverability within PRINCE. Due to historical data migrations and varied annotation practices across different laboratories and systems over Bayer's extensive operational history, the metadata can sometimes be incomplete, missing, or incorrect. To address this challenge and continuously enhance the quality of the structured metadata, we have developed a utility system that employs Named Entity Recognition (NER) to extract and create accurate annotations directly from the study PDFs. This system is designed to read the textual content of the preclinical reports and identify key entities and associated information that should be represented in the structured metadata. The process involves: - Processing study PDFs to extract text and identify relevant entities (e.g., study IDs, compound names, species, routes of administration, dosage information, clinical findings, etc.). - Generating structured annotations based on the identified entities and their relationships within the text. We are actively working on integrating this utility system into our data pipelines to automatically correct and enrich the data within the Amazon Athena database. The system's performance in generating accurate annotations has been evaluated against curated datasets, demonstrating promising results. To manage the integration of these annotations into the production database, we are developing an evaluation system that provides a confidence score for each extracted field. Fields with a high confidence score will be automatically used to update the corresponding entries in Amazon Athena. Fields with lower confidence scores will be quarantined and flagged for human review and intervention, ensuring data accuracy while leveraging automation. This approach aims to continuously improve the quality of the structured metadata, making it a more reliable source of information for PRINCE and other downstream applications. The Journey Continues: Iterative Development PRINCE has been available to end-users since early 2024, with the agentic integration introduced later that year. This has been crucial for gathering real-world feedback and driving iterative development. A key principle guiding our development has been the understanding that building a production-ready LLM application is an iterative process; we don't wait for features to be absolutely perfect before seeking user feedback. Instead, we prioritize delivering value early and continuously refining the system based on real-world usage. In the initial stages, our focus was squarely on achieving the desired accuracy and performance for core functionalities, even if it meant incurring higher costs. We recognized that optimizing for cost prematurely could compromise the system's effectiveness and hinder user adoption. Only after achieving the desired level of accuracy and performance did we begin to focus on cost optimization, ensuring that efficiency gains did not negatively impact the user experience or the quality of the results. The development of PRINCE follows a continuous, iterative process. User feedback, ongoing monitoring data, and insights from expert scientists are continuously fed back into the development cycle, leading to refinements in the architecture, retrieval techniques, agent behaviors, and user interface to enhance performance, usability, and ultimately, scientific impact. Conclusion Building a production-ready LLM application in a complex enterprise environment like preclinical drug discovery is a journey marked by significant technical and engineering challenges. The PRINCE case study demonstrates that by combining robust data infrastructure, sophisticated information retrieval techniques like RAG and Text-to-SQL, and an intelligent multi-agent orchestration system, it is possible to unlock valuable insights from vast, previously inaccessible data repositories. Our experience highlights the critical importance of focusing on engineering for reliability, including robust error handling, state persistence, and LLM fallbacks. Furthermore, building user trust is paramount, achieved through transparency in the workflow, clear explainability via granular citations, and continuous evaluation and monitoring of the system's performance. PRINCE has already shown promising results in enhancing data accessibility and research efficiency at Bayer, transforming how scientists interact with preclinical information. This is not the end of the journey, but rather a significant step towards creating truly intelligent research assistants. The broader lesson from PRINCE is that production-ready agentic AI is not only about better models or better prompts. Reliability comes from engineering both the context the model sees and the harness within which the model acts. Context engineering helped ensure that each model had the right information, and only the right information, at the right stage of the workflow. Harness engineering helped ensure that the workflow remained bounded, observable, recoverable, and suitable for a regulated research environment. As model capabilities improve, some parts of today's harness may become thinner or move into native model capabilities. But in enterprise research systems, especially where trust, traceability, and reviewability matter, explicit control over context, workflow state, recovery, reflection, and verification remains essential. We hope this overview provides valuable insights into the practical considerations and technical depth required to build and productionise LLM applications in a regulated and data-rich domain. Acknowledgments The author gratefully acknowledges the invaluable contributions of Adam Zalewski, Annika Kreuchwig, Carlos Henrique Vieira-Vieira, Jobst Löffler, and Jonas Münch from the Bayer team. The author also thanks Bala Hari, Balu Saravanan, Bernice Mercy Sharon M, Deril X, Jigar Jani, Manibalan Baskaran, Nafis Aslam, Priyalakshmi R, Rohit Bansal, Sai Prabhanj Turaga, Saksham Srivastava, Shivam Sehgal, Sowmya Adimoulame, and Subhashini Rajamani from the Thoughtworks team for their contributions to this work. The author used AI assistance during the writing of this article. AI tools were used for brainstorming ideas, creating outlines, and reviewing drafts to polish language and improve clarity. Disclaimer All activities described conform to Bayer's information classification, data governance, and external communication policies, and do not constitute claims regarding regulatory decision‑making or product performance. Significant Revisions 16 June 2026: published

When I reject AI code even if it works With implementation getting faster and faster, the real bottleneck moves to reviewing the volume of code generated by AI. I’m not even talking about your coworkers’ (and their agents’) PRs, but your own git diff after your coding agent has finished its job. Even when I follow good practices – like starting with the plan mode, dividing big tasks into phases, and shipping small changes – I still feel cognitive overload when reviewing something I haven’t actually thought through myself. Before coding agents, when given a task, I would explore the codebase, think of different solutions, experiment, and only then implement. That could take days of consolidating all that context. When I finally submitted that PR, confidence was higher, and explaining each of my changes to my coworkers was easier. I have to admit that with AI, completing big tasks still takes me days. More often than not, I reject all changes made by AI and start over. The difference between the first session and the second is not the LLM model, but the person behind the screen. With more time to consolidate the problem I’m trying to solve, I can drive the agent to a better solution instead of being driven by it. More and more, I reject AI code for the same reasons: - I reject AI code when I can’t explain the approach in my own words. - I reject AI code when the diff is bigger than the problem. - I reject AI code when it introduces abstractions before proving they’re needed. - I reject AI code when it works locally but makes the system harder to reason about. - I reject AI code when I’m trusting the output more than my understanding. It’s not uncommon to see engineers accept AI-generated changes too quickly, and that is why I advocate for required human review in conjunction with AI reviews. The reality is that code that runs and makes the CI green can still be a bad solution, and engineering has always been about implementing adequate, scalable, and extensible solutions. I’ve been using coding agents for some time, and despite how impressive they are, they still need a great engineer guiding them to great solutions. Yes, coding agents can help you with this task with more than just writing code, but that doesn’t mean they can do it autonomously in a sustainable manner yet.

State of Unreal UE6 Reactions: Hype, Skepticism, and What It Means for Game Devs Developers and industry experts react to Epic’s State of Unreal announcements, from UE 5.8 and UE6 to MCP, AI-assisted workflows, Verse, and the future of Blueprints. In case you missed it UE6 Updates Industry experts and Unreal Engine developers are reacting to Epic Games’ latest State of Unreal announcements, including Unreal Engine 5.8, early details on UE6, MCP-powered AI workflows, Verse, and the eventual transition away from Blueprints. While some see the updates as a major step toward theoretically beneficial, AI-assisted development, others are raising concerns about accessibility, authorship, traditional art pipelines, and what these changes mean for teams built around existing Unreal workflows. Positive Sentiments from State of Unreal 2026 For many developers and Unreal Engine users, the biggest takeaway from State of Unreal was that Epic is not simply treating UE 5.8 and UE6 as another round of renderer, workflow, or performance updates. The company appears to be laying the groundwork for a broader production shift around AI-assisted workflows, Verse, interoperable content, and a more unified relationship between Unreal Engine and UEFN. In other words, they're making big strides towards building a Metaverse Engine, which has clearly been the plan by Epic for many years. In its official roadmap, Epic said UE6 is meant to improve rendering, reduce cook times, tighten iteration loops, and make mobile increasingly capable, while also changing how games are shipped and operated through Verse, open standards, interoperable content, and MCP integrations for tools like Claude and Gemini. First and foremost, the slew of exciting features outlined for Unreal Engine 5.8 has lots of developers excited. Mesh Terrain seems extremely impressive, Lumen Light for lower-spec devices such as Switch 2 is a great signal for optimization, upcoming UE5 games like No Law look impressive, and version control via Lore has lots of upside. Managing Director at Reynire Virtual, Bjørne Hoff, sees a lot of upside with what's coming next via UE6: "Oh Lots of interesting news at State of Unreal...Mesh terrain is world terrain upgraded with Boolean tools among other things. My favourite bit of the show! We’ll get the real deal in a year and a bit. Controversial: UEFN and UE is going to merge into the same suite when UE6 releases. Honorable mention to the new Depth of Field system to cinematics and virtual production people." - Bjørne Hoff, Managing Director at Reynire Virtual Many developers note the matter-of-fact nature of this year's State of Unreal has an air of inevitability behind it all, as well as a certain clarity of vision that's appreciated. Former Epic Games Technical Program Manager and current Lead Visualization Artist at [namethemachine], Brian J. Pohl, characterizes this week's news as a key "turning point" for the industry due to the focus on AI integrations: Some developers responded positively to that ecosystem-level ambition. Matthew J.X. Doyle, Studio Director at Spider Lily-creator falsework, described the “real throughline” of the keynote as UE5 and UEFN converging into a foundations-first engine built on Verse, distributed software transactional memory, open standards, and portable, interoperable content. "There's a lot to unpack – including the exciting announcement of Lore, a new open-source version control system for Unreal – but, the real throughline was the UE6 vision — UE5 and UEFN converging into a single, foundations-first engine built on Verse and distributed software transactional memory, with a serious bet on open standards and portable, interoperable content, all in service of Tim's thesis that the future of games is an open, interconnected ecosystem developers build together as peers, with no overlord." - Matthew J.X. Doyle, Studio Director at falsework According to Unreal evangelist, XR leader, Unreal NYC Lead Organizer, and Agile Lens CEO Alex Coulombe, Epic shipping an official MCP plugin is a sign of the company "planting a flag" to establish associated plugins as a core part of the engine going forward. One of the most contentious parts of the UE6 news is that, eventually, Actros and Blueprints will be deprecated in favor of the new Verse programming structure leveraging Scene Graph. The idea is that this will replace Blueprints and be an improvement—in theory. "Non-visual scripting languages are generally simpler to maintain in terms of execution environment, optimization. They are also a lot more mergeable, which makes it easier to work with in bigger teams. My best guess is that they're banking on LLMs being relatively widely available throughout the lifecycle of UE6. If you package enough context for a tight-scoped scripting language, you can probably create comparably convenient interface for non-programmers for the purposes of scripting. Imagine old-school LUA-based scripting, but with a programmer next to you all the time. That would probably be the idea. And it will likely be easier to point-optimize later for specialist human programmers into C++ as well." - Anton Mykailenko, Co-Founder/CEO of Keyboard Cats Studio as a comment on this LinkedIn post from Chris Geisler-Johnson Taken together, the optimistic reactions suggest that many developers see State of Unreal as more than a feature showcase. To them, Epic is trying to solve production problems like fragmented workflows between UE5 and UEFN, the difficulty of scaling live and persistent worlds, the pain of collaboration around assets and code, and the amount of manual work that slows teams down. As of now, positive sentiment is focused on new UE 5.8 features and how UE6 could make Unreal more connected, more interoperable, and more responsive to how modern teams actually build and ship games. Skeptical Sentiments from State of Unreal 2026 But not every reaction to State of Unreal was optimistic. While Epic positioned UE 5.8 and UE6 around faster iteration, AI-assisted workflows, Verse, MCP integrations, and a more unified Unreal/UEFN ecosystem, some developers and artists saw the same announcements as signs of a more disruptive transition than Epic’s framing suggested. The clearest practical concern is the future of Blueprints. In its official UE6 roadmap, Epic said Actors and Blueprints will remain in early versions of UE6, but will “eventually” be deprecated once the new framework is mature, with conversion tools planned to help move projects over. Epic also said UE6 Early Access is targeted for the end of 2027, with the full release expected 12–18 months later, meaning this is not an immediate break, but it is a clear long-term direction. That detail immediately raised questions because Blueprints have been one of Unreal Engine’s defining accessibility features for years. In fact, Game of the Year-winning Clair Obscur: Expedition 33 was made possible because its small team of developers built all of its gameplay systems using Blueprints. Ataberk Ateş, Lead Technical Designer at Climax Studios, theorizes this push towards a "unified engine" is predicated on the desire to actually evolve Unreal into a "metaverse engine" instead: In the comments of Alex Coulombe's post about State of Unreal, one technical artist explains why he is "definitely not a fan" of these announcements as they relate to the AI tooling: "Honestly, I hate this. Definitely not a fan. Making fun little environments and exploring the creation process for cozy environments, creating shaders to fit the scene, and directing the lighting to fit a mood have been such a joy for me. Sharing that work at the end of the process is extremely fulfilling. The joy in creation does not come from the volume of output. It comes from the art process. Tools like this exist to remove the creation process entirely. Now if I were to go back and make one of those cozy environments, everyone will just assume I used AI tools like this." - Steve Biegun, Senior Technical Artist at Industrial Light & Magic According to BAFTA Member and Ex-Epic Games, Infinity Ward, and Rebellion UI Artist Chris Geisler-Johnson, the move away from Blueprints after years of foundational work is "risky" due to how Blueprints "lower the entry barrier" for any game developer. Epic’s roadmap does try to soften that concern, saying its philosophy is to bring existing projects along rather than “force a hard break.” But even with that reassurance, the wording still confirms a fundamental shift: UE6’s new gameplay framework, Scene Graph, is being built from scratch on Verse, so time will tell what that sort of upheaval and change will be like for in-development projects seeking to move over to UE6. What It Means for Game Devs Going Forward Since UE6 is being positioned as a unified creation ecosystem that brings together Unreal Engine 5 and UEFN, with Tim Sweeney describing the vision as “UE5 plus UEFN equals UE6,” it's clear there are some big changes on the horizon. The goal is not just to improve rendering or performance, but to let developers build once and ship across PC, console, mobile, Fortnite, and potentially other UE6-based ecosystems. That could be a major opportunity for studios, especially smaller teams and live-service developers, because it points toward faster iteration, broader distribution, and more portable content. Epic’s official UE6 roadmap says the company wants to improve iteration loops, reduce cook times, expand open standards, and use Verse as the foundation for a new programming model. In theory, that means teams could spend less time fighting fragmented pipelines and more time building content that can move across platforms and ecosystems more easily. The broader implication is that Unreal development may become more powerful, but also more strategically complicated. AI-assisted editor control, Verse-based systems, interoperable content, and Fortnite-connected distribution could open new doors, but they also raise practical questions around training, maintainability, authorship, team roles, and production policy. What about you? What do you think? Subscribe to 80 Level Newsletters Latest news, hand-picked articles, and updates Don't forget to subscribe to our Newsletter, join our 80 Level Talent platform, follow us on Twitter, LinkedIn, Telegram, and Instagram, where we share breakdowns, the latest news, awesome artworks, and more. Are you a fan of what we do here at 80 Level? Then make sure to set us as a Preferred Source on Google to see more of our content in your feed.

I just taught an experimental class at the University of Pennsylvania where I challenged students to create a startup from scratch in four days. Most of the people in the class were in the executive MBA program, so they were taking classes while also working as doctors, managers, or leaders in a variety of large and small companies. Few had ever coded. I introduced them to Claude Code and Google Antigravity, which they needed to use to build a working prototype. But a prototype alone is not a startup, so they used ChatGPT, Claude, and Gemini to accelerate the idea generation, market research, competitive positioning, pitching, and financial modelling processes. I was curious how far they could get in such a short time. It turns out they got very far.

I’ve been teaching entrepreneurship for a decade and a half, and I've seen thousands of startup ideas (some of which turned into large companies) so I have a good sense of the expectations for what a class of smart MBA students can accomplish. I would estimate that what I saw in a couple of days was an order of magnitude further along the path to a real startup than I had seen out of students working over a full semester before AI. Most of the prototypes were not just sample screens but actually had a core feature working. Ideas were far more diverse and interesting than usual. Market and customer analyses were insightful. It was really impressive. These were not yet working startups nor were they fully operational products (with a couple exceptions) — but they had shaved months and huge amounts of money and effort from the traditional process. And there was something else: most early startups need to pivot, changing direction as they learn more about what the market wants and what is technically possible. By lowering the costs of pivoting, it was much easier to explore the possibilities without being locked in or even explore multiple startups at once: you just tell the AI what you want.

I wish I could say this impressive output was the result of my brilliant teaching, but we don’t really have a great framework yet for how to use all these tools, the students largely figured it out on their own. It helped that they had some management and subject matter expertise because it turns out that the key to success was actually the last bit of the previous paragraph: telling the AI what you want. As AIs are increasingly capable of tasks that would take a human hours to do, and as evaluating those results becomes increasingly time consuming, the value of being good at delegation increases. But when should you delegate to AI?

The Equation of Agentic Work

We actually have an answer, but it is a bit complicated. Consider three factors: First, because of the Jagged Frontier of AI ability, you don’t reliably know what the AI will be good or bad at on complex tasks. Second, whether the AI is good or bad, it is definitely fast. It produces work in minutes that would take many hours for a human to do. Third, it is cheap (relative to professional wages), and it doesn’t mind if you generate multiple versions and throw most of them away.

These three factors mean that deciding to delegate to AI depends on three variables:

1. Human Baseline Time: how long the task would take you to do yourself

2. Probability of Success: how likely the AI is to produce an output that meets your bar on a given attempt

3. AI Process Time: how long it takes you to request, wait for, and evaluate an AI output

A useful mental model is that you’re trading off “doing the whole task” (Human Baseline Time) against “paying the overhead cost” (AI Process Time), possibly multiple times until you get something acceptable. The higher Probability of Success is, the fewer times you have to pay AI Process Time, and the more useful it is to turn things over to the AI. For example, consider a task that takes you an hour to do, but the AI can do it in minutes, though checking the answer takes thirty minutes. In that case, you should only give the work to the AI if Probability of Success is very high, otherwise you’ll spend more time generating and checking drafts than just doing it yourself. If the Human Baseline Time is 10 hours, though, it could be worth several hours of working with the AI, assuming that the AI can be made to do a competent job.

We know this equation works because this past summer, OpenAI released one of the more important papers on AI and real work, GDPval. I have discussed it before, but the key was that it pitted experienced human experts in diverse fields from finance to medicine to government against the latest AIs, with another set of experts working as judges. It took experts seven hours on average to do the work, so, in this case, that is the Human Baseline Time. The AI Process Time was interesting: the AI took only minutes for tasks, but it required an hour for experts to actually check the work, and, of course, prompts take time to write as well. As for Probability of Success, when GDPval first came out, judges gave human work the win the majority of the time, but, with the release of GPT-5.2, the balance shifted. GPT-5.2 Thinking and Pro models tied or beat human experts an average of 72% of the time.

We can now calculate how many hours you would save on a seven-hour task, assuming that 72% probability of success and an hour of evaluation. If you tried every task by taking the time to prompt the AI, evaluating the answer for an hour, and then doing it yourself if the AI answer was bad, you would save 3 hours on average. Tasks the AI failed on would take longer (you wasted time prompting and reviewing!) but tasks the AI succeeded on would be much faster. But we can change the equation even more in our favor using techniques from management!

Delegation as the new prompting

There are three things we can do to make delegating to AI more worthwhile by increasing the Probability of Success and lowering AI Process Time. We can give better instructions, setting clear goals that the AI can execute on with a higher chance of succeeding. We can get better at evaluation and feedback, so we need to make fewer attempts to get the AI to do the right thing. And we can make it easier to evaluate whether the AI is good or bad at a task without spending as much time. All of these factors are improved by subject matter expertise — an expert knows what instructions to give, they can better see when something goes wrong, and they are better at correcting it.

If you don’t need something specific, AI models have become incredibly capable of figuring out how to solve problems themselves. For example, I found Claude Code was able to generate an entire 1980s style adventure game with one prompt to "create an entirely original old-school Sierra style adventure game with EGA-like graphics. You should use your image agent to generate images and give me a parser. Make all puzzles interesting and solvable. Finish the game (it should take 10-15 minutes to play), don’t ask any questions. make it amazing and delightful." That’s it, the AI made everything, including the art. With two final prompts it tested the game and deployed it. You can play it yourself: enchanted-lighthouse-game.netlify.app

This is genuinely amazing, but that amazement is amplified because I didn’t need anything specific, just an adventure game that the AI was free to improvise. But real work, and real delegation, means that you have a specific output in mind, and that is where things can get tricky. How do you communicate your intention to the AI to execute on what you want, so it can use “judgement” to solve problems while still giving you the output you desire?

This problem existed long before AI and is so universal that every field has invented their own paperwork to solve it. Software developers write Product Requirements Documents. Film directors hand off shot lists. Architects create design intent documents. The Marines use Five Paragraph Orders (situation, mission, execution, administration, command). Consultants scope engagements with detailed deliverable specs. All of these documents work remarkably well as AI prompts for this new world of agentic work (and the AI can handle many pages of instructions at a time). The reason you can use so many formats to instruct AI is that all of these are really the same thing: attempts to get what’s in one person’s head into someone else’s actions.

When you look at what actually goes into good delegation documentation, it’s remarkably consistent: What are we trying to accomplish, and why? Where are the limits of the delegated authority? What does “done” look like? What specific outputs do I need? What interim outputs do I need to follow your progress? And what should you check before telling me you’re finished? If these are well-specified, the AI, like humans, is far more likely to do a good job.

And in figuring out how to give these instructions to the AI, it turns out you are basically reinventing management.

Managing Agents

I find it interesting to watch as some of the most well-known software developers at the major AI labs note how their jobs are changing from mostly programming to mostly management of AI agents. Coding has always had a very organized structure, with clearly verifiable outputs (the code either works or it doesn’t) so it has been one of the first areas where AI tools have matured, and thus the first profession to feel this change. It isn’t the last.

As a business school professor, I think many people have the skills they need, or can learn them, in order to work with AI agents - they are management 101 skills. If you can explain what you need, give effective feedback, and design ways of evaluating work, you are going to be able to work with agents. In many ways, at least in your area of expertise, it is much easier than trying to design clever prompts to help you get work done, as it is more like working with people. At the same time, management has always assumed scarcity: you delegate because you can’t do everything yourself, and because talent is limited and expensive. AI changes the equation. Now the “talent” is abundant and cheap. What’s scarce is knowing what to ask for.

This is why my students did so well. They weren’t AI experts. But they’d spent years learning how to scope problems in their fields of expertise, define deliverables, and recognize when a financial model or medical report was off. They had hard-earned frameworks from classes and jobs, and those frameworks became their prompts. The skills that are so often dismissed as “soft” turned out to be the hard ones.

I don’t know exactly what work looks like when everyone is a manager with an army of tireless agents. But I suspect the people who thrive will be the ones who know what good looks like — and can explain it clearly enough that even an AI can deliver it. My students figured this out in four days. Not because they were AI natives, but because they already knew how to manage. All that training, it turns out, was accidentally preparing them for exactly this moment.

Subscribe now

Share

19th June 2026 The real valuable capability MCP offers over skills/CLI is isolating the auth flow outside of the agent’s context window, and potentially out of the harness completely. [...] Maybe the idealized form of MCP is just an auth gateway for the API and nothing else. That’d still be a win. — Sean Lynch, comment on Hacker News Recent articles - Datasette Apps: Host custom HTML applications inside Datasette - 18th June 2026 - GLM-5.2 is probably the most powerful text-only open weights LLM - 17th June 2026 - Publishing WASM wheels to PyPI for use with Pyodide - 13th June 2026

17th June 2026 What happened in 2025 was this: the economics of code production were turned upside down. Instead of being very hard, time-consuming, and expensive to generate code, it became effectively free and instant. Lines of code went from being treasured, reused, cared for and carefully curated, to being disposable and regenerable, practically overnight. — Charity Majors, AI demands more engineering discipline. Not less Recent articles - Datasette Apps: Host custom HTML applications inside Datasette - 18th June 2026 - GLM-5.2 is probably the most powerful text-only open weights LLM - 17th June 2026 - Publishing WASM wheels to PyPI for use with Pyodide - 13th June 2026