This seminar discussed what kind of computing, semiconductor, memory, software, and policy strategies Korea should prepare in a situation where AI has entered the era of 'Agentic AI', which goes beyond simply being a tool to answer questions, plans on its own, cooperates with multiple agents, and performs actual tasks. At the core, it was emphasized that agentic AI requires significantly more tokens, memory, power, communication, and data center resources than existing AI, and that the existing GPU-centered infrastructure strategy must also change significantly accordingly. The panelists agreed that Korea needs a strategy that builds on its memory strengths, but also protects full-stack capabilities and a self-sustaining ecosystem that connects models, hardware, software, and services, as well as human judgment and basic education.

1. The advent of the agentic AI era and the purpose of the seminar

Kim Seung-tae, Director of AI Infrastructure Support Bureau, explained in his opening speech that this seminar was prepared to discuss national AI computing strategies in connection with the three mega projects recently announced by the government. The purpose was to share with experts what has been discussed in the Technology Innovation and Infrastructure Division and to jointly explore the direction of AI development in Korea.

Shin Jin-woo, Chairman of the Technology Innovation and Infrastructure Subcommittee, said that the pace of recent AI technology development is "different from yesterday and today," and diagnosed that while not long ago the focus was on super AI competition that creates larger models, we are now moving into the Agent AI Era where AI makes plans on its own and uses tools to handle actual tasks.

"Not long ago, the focus of competition was on creating super-large AI and larger models, but now we seem to have entered the era of agentic AI, where AI makes plans on its own and uses tools to perform actual tasks."

Global competition is also rapidly intensifying. It was mentioned that while the competition for AI hegemony is intensifying, centered on the United States and China, Antropic made a strong advance and China's Moonshot AI released Kimi K3, which shows performance close to the top model at a low cost, causing a great shock to the market.

However, national AI competitiveness is not completed by securing just one excellent model. Chairman Shin-woo Shin explained that for the model to operate stably and efficiently, computing infrastructure, data centers, semiconductors, networks, power, and policy must develop together.

"Only when technology, infrastructure, and policy evolve together can national AI competitiveness be secured."

Korea is also pursuing various policies, including building a national AI data center, creating an AI semiconductor ecosystem, and securing physical AI technology, with the goal of becoming one of the top three AI powers. This seminar was an opportunity to ask what choices Korea should make in the field of AI technology and infrastructure amidst this trend.

2. The rules of the game change from language model to agentic AI

Lee Dong-soo, a member of the Technology Innovation and Infrastructure Subcommittee, emphasized in his presentation that rather than simply comparing "which model had higher performance," we should pay attention to how agentic AI changes the game rules of industry and computing itself.

The existing language model was to provide one answer when the user entered a question. However, agentic AI involves multiple specialized agents communicating with each other, using external tools, and verifying results to perform specific tasks. If the result is wrong, analyze the cause of the failure, then go back to the previous step and try again.

"It has developed into a complex system that can be repeated dozens or hundreds of times until the initially given business goal can be achieved."

For example, a coding agent does not have one model that handles all tasks, but operates in a structure where the agent that writes the code, the agent that verifies the results, and the agent that analyzes errors cooperate with each other. In this process, agents share messages and intermediate results and utilize external programs, databases, and search tools.

Commissioner Lee Dong-soo compared the existing language model to the PC era and agentic AI to the Internet age. While the existing model was an independent model where people directly input data and receive answers, agentic AI is a network-type structure in which multiple systems are connected and cooperate.

Agentic AI implemented in everyday life

Agentic AI is already appearing in a form close to the future AI we have imagined. If you say "Please send an email to someone" while driving, functions such as voice recognition, email writing, contact search, and schedule checking can work together to accomplish the task.

The same goes for when there are no eggs in the refrigerator, AI recognizes this, searches the shopping mall, and processes orders and payments. In this case, the refrigerator agent, shopping mall agent, and payment agent are linked.

"Now, the characteristics of agentic AI are not simply creating tokens, but also knowing how to cooperate and what tools to use to achieve the goal."

The fields of use are very wide, including voice interface, commerce, coding, finance, manufacturing, and personal assistant. However, as the scope of use expands, the burden on infrastructure also increases rapidly.

3. The enormous cost and infrastructure required by agentic AI

Agentic AI requires much more computing resources than a typical language model. Rather than answering a single question once, multiple agents talk multiple times, revise the plan, verify the results, and execute again if it fails.

Commissioner Lee Dong-soo explained that the cost of agentic AI infrastructure is likely to be 100 to 1,000 times larger than existing language models. Several factors act like multiplication here.

The first is increased request volume. In the future, rather than directly operating the app, users will issue abstract commands such as "Book a trip" or "Reschedule a meeting." AI then takes over multiple applications and services, significantly increasing overall traffic.

The second is increased token usage. Agents think on their own, make plans, and perform multi-step tasks. If the task fails, it goes back to the previous step and re-executes, so much more tokens are needed than a simple question.

"The agent must think on its own and take several turns. If it fails, it goes back and repeats, so it can create more than 10 times more tokens than a normal language model."

Third is increased memory and infrastructure costs. Since multiple agents must use different models and read and store each other's intermediate results, not only GPU but also large memory, storage, and network become important.

Some studies have also shown that the power required to answer one question can be 136 times more than existing language models. This is because the agent repeats inferences dozens of times and switches between multiple models and tools.

Hardware and software innovations that offset cost increases

However, costs do not just continue to rise. Technologies that increase efficiency in both hardware and software are also rapidly emerging.

Representative examples include multi-token prediction and specular decoding. Previously, one token was created at a time, but this method increases the creation speed by predicting multiple tokens in advance. There are cases where costs can be reduced by about 4.4 times using these software techniques alone.

On the hardware side, modern GPUs can process more tokens per power than previous generations. Applying software and hardware innovations together has the potential to reduce costs by more than 10 times.

"There are areas where costs rise rapidly by 100 or 1,000 times as agent AI progresses, but competition will continue to lower costs by tens or hundreds of times through innovation in hardware and software."

For this reason, the world's big tech companies are investing heavily in data centers and GPUs to respond to agentic AI and inference services. Although cash flow will rapidly deteriorate in the beginning, the analysis is that expectations that inference services will generate full-fledged sales and profits in two to three years are driving investment.

4. From GPU-centered to memory/communication-centered infrastructure

In the existing AI infrastructure, the GPU and HBM, a high-speed memory close to the GPU, were key. By deploying the same model to multiple GPUs, there was no problem which GPU processed the request, and by distributing work to idle GPUs, efficiency could be increased.

However, in agentic AI, several types of specialized models and agents operate simultaneously. Since the required model is different for each request, the task cannot be sent to any GPU, but must be sent to a specific GPU or model. This may result in lower GPU utilization.

A more important change is that agents must read each other's intermediate results. When a message or state created by one agent is stored in memory, another agent reads it and performs the next task. Therefore, large memory, storage, and communication between memories become as important as the computational performance of the GPU.

Commissioner Lee Dong-soo explained that this change is similar to the emergence of smartphones. When cell phones were simple call, music, and game devices, each function operated independently, but in the smartphone era, multiple apps share a single memory and operating system.

"Agentive AI is like the iPhone moment. Just as multiple apps needed an operating system and shared memory to work simultaneously, multiple agents need new coordination layers and memory management to work together."

In agentic AI, GPU and HBM alone are not sufficient, and a communication layer or coordination layer that coordinates cooperation between agents is needed. This allows the results created by one agent to be retrieved directly from memory without being recalculated by another agent.

Data center evaluation criteria must also change

For existing AI data centers, the key evaluation criterion was how quickly and cheaply they could produce input tokens and output tokens. However, in agentic AI, one task is repeated multiple times and communication between agents also occurs.

Therefore, rather than simply "how many tokens are generated per power", we need high-level metrics such as:

• The total cost of completing a task
• Time it takes to complete the task
• Communication costs between agents
• Number of failures and retries
• Success rate of achieving the goal
• Actual throughput per power and memory

"Agent AI should not only look at the price per token, but also evaluate how efficiently a given goal is achieved."

In fact, a model with a low token price may be more expensive for the overall operation. This is because if an inexpensive model fails multiple times or works over a long period of time, the total cost can be greater than a model that processes correctly at once, even if the unit price is expensive.

5. Harness and context management are more important than models

The performance and cost of agentic AI vary greatly depending on the design of the harness as well as the model itself. Harness is a type of task management structure that allows AI to decide how to perform a task, where to store intermediate results, which agent to call, and how to retry in case of failure.

Even if the same model is used, the cost can be more than double depending on how the harness is designed. In particular, how to share intermediate states and memories between agents is important.

Context issues, such as emails continuing

If an agent communicates multiple times, previous input and output continue to accumulate. This is similar to receiving dozens of reply emails at work. There are simple work instructions at the top, but in reality, you have to read all the long emails piled up below.

"An agent's conversation is like an email with 20 or 30 replies. There are simple instructions at the top, but it can take two hours to refer to what's underneath."

If the context becomes too long, processing speed and cost deteriorate rapidly. To solve this problem, current agentic AI platforms compress or summarize the content of conversations. However, during this process, important information may be lost or AI may misunderstand the context.

In fact, in one case, an incident occurred where the agent deleted 200 of the user's emails because it misrecognized the email content during the context compression process. The fact that it is difficult for agents to stop work immediately after starting work was also presented as a risk factor.

It has been evaluated that the current agentic AI is not yet a completed system, but is closer to the stage of heuristic adjustment between a slow but relatively high success rate method and a fast and inexpensive method with a possibility of failure.

6. Importance of inference models, KV cache and next-generation memory

In agentic AI, an analysis was also introduced showing that inference/reasoning models can actually reduce overall costs compared to general models. Inference models use more thought tokens to create a single response, but the overall number of operations can be reduced because they identify problems more accurately and reduce failures.

"Thoughtful reasoning models can actually use fewer tokens overall because they have a clear understanding of what to do next and reduce failures and retries."

In other words, you should not simply look at the model's token price or one-time response speed. You need to evaluate how quickly and with little trial and error the entire task can be completed.

KV cache and memory access

In agentic AI, especially coding agents, most of the preceding dialogue and code context is already calculated. Rather than having to recalculate the entire content on the GPU each time, it may be more efficient to load the previous calculation results, KV Cache, from memory.

The presentation explained that when processing the same 10,000 tokens of input, fetching the cache from storage can be more than 10 times faster than re-performing the prefill operation on the GPU. This is why the service price is greatly reduced when the cache is hit.

"Rather than having to recalculate the previous message on the GPU, it is faster to retrieve the encoding result stored in memory."

Accordingly, in agentic AI, a structure that quickly reads and shares data from memory may become more important than a structure in which the GPU continuously calculates. This is because the prefill stage ends quickly, and in the decode stage, memory access can become a bottleneck rather than the GPU's calculation unit.

This analysis forces us to fundamentally reexamine the design of existing AI infrastructure.

"There is even talk that we may need to rethink the AI ​​infrastructure we have known so far from the beginning."

New MPU and memory architecture

For future AI semiconductors, or MPUs, it is not enough to simply be good at matrix operations. It must coordinate various agents and memory, handle heterogeneous services, and support new memory technologies such as CXL or high-capacity flash-based memory.

However, there is a question of 'which came first, the chicken or the egg?' Memory devices are developed only when the host chip supports new memory, but if market demand is not certain, it is difficult for host chip companies to add functions. Conversely, without a host chip, it is difficult for memory companies to conduct proper experiments.

Commissioner Lee believed that this change could actually be an opportunity for startups. This is because they can analyze new workloads faster than large companies and design hardware and software together.

7. Full-stack strategy that designs hardware and software together

One of the key conclusions of the presentation was that software and hardware should not be developed separately.

For example, using a tuning method such as LoRA to grant specialized functions to each agent can increase model performance, but may make it difficult to share the KV cache between agents. Then, the previous calculations must be redone, slowing down the process.

The researchers first set a hardware constraint that the KV cache must be shared, and then experimented with a method to maintain LoRA tuning performance within that condition. As a result, it was possible to maintain many of the specialized functions while forcing cache sharing, and the response speed was improved by 11 times and throughput by about 4 times.

"When hardware constraints are combined with algorithms, even the same algorithm can make a big difference in cost and speed."

In this way, when developing memory technology, we must not only look at "which memory is faster," but also consider what memory structure is needed in actual services and what algorithm it will be combined with.

"We must not stop at saying that we are a memory powerhouse, but we must also be able to explain what service innovations are possible by using that memory."

To achieve this, the following agent-aware infrastructure and infrastructure-aware agent are required.

• Designed so that agents can utilize the limitations and advantages of hardware
• Designed so that the infrastructure understands the agent's work characteristics
• Establishment of agent scheduler and memory management function
• Development of inference operating system for agents
• Connecting next-generation memory and semiconductors to the software ecosystem

Korea has strengths in the memory field, including HBM, CXL, HBF, and next-generation DDR. However, it is pointed out that AI can lead to competitiveness if it goes beyond simply supplying parts and develops services and systems that utilize it.

8. Korea's AI 3rd party strategy and national AI sovereignty

The first topic of the panel discussion was what South Korea must do to transform its memory strengths into AI competitiveness.

Professor Lee Ki-min of KAIST said that in the era of agentic AI, multi-turn dialogue, memory utilization, quick response, and the ability to coordinate multiple agents become more important than models that simply think longer. Therefore, he explained that a specialized agentic AI model that is different from the existing model must be developed, and the computer architecture and operating system must also be changed accordingly.

Lee Yun-soo, vice president of Samsung Electronics, emphasized that if Korea has so far participated as a partial component in the big game created by Nvidia or Big Tech, it must be redesigned end-to-end from chips to services so that it can lead the entire game in the future.

"Rather than being a part of someone else's plan, we need a complete redesign so that we can take the lead."

Commissioner Lee Dong-soo said that Korea needs to be bold again like it did in the past when it pioneered the memory and shipbuilding industries. Currently, there is a strong tendency to check whether other companies are successful first and see if there is a return on investment before moving on, but in order to create a new industry, you must actively communicate with the market and find needs.

"The answer to what will sell well ultimately lies in the service market."

He emphasized that semiconductor design services should start from a deep understanding of the actual use environment. This means that you must constantly check which features are actually needed, why users are inconvenienced, and why the product is not chosen by the market.

Need to see the whole stack, but need to prioritize

There were various opinions on whether the foundation model, GPU data center, memory, inference optimization, and agentic software should all be done directly to secure national AI sovereignty.

Commissioner Lee Dong-soo ultimately argued that the entire stack must be viewed. This is because excessive reliance on models and GPUs from specific countries or companies can cause the domestic AI industry to come to a halt when export regulations or geopolitical changes occur. However, he added that it is difficult to become the world's best in all fields in a short period of time, so setting strategic priorities is necessary.

Professor Lee Ki-min also said that he was skeptical about the full-stack strategy at first, but his thoughts have recently changed. This is why full stack is not necessarily a strategy to become number one in the world in cutting-edge models, and it is important to at least have the ability to survive and become self-reliant.

"You don't just need a full stack to go to Frontier Lab, it's important that we have the ability to be self-sufficient to some extent."

9. National policy must create an ecosystem and flywheel

In the discussion, opinions were divided over whether the country should directly create AI demand or only support the supply side, such as GPU, MPU, and model development.

Professor Lee Ki-min believed that if a good model and inexpensive service come out first, demand can naturally arise. If you force demand, there is a possibility that it will become a perfunctory business with low actual value.

Vice President Lee Yoon-soo said that the government should be cautious in creating all demand on behalf of the market, but the government should take charge of nurturing talent, creating a research base, and easing excessive regulations, which are difficult for companies to do on their own.

On the other hand, Commissioner Lee Dong-soo believed that a certain level of priming was needed to form the initial ecosystem. The explanation is that, like the US DARPA project or China's large-scale state support, the country must initially invest to create a success story and let the market run on its own.

Director Park Tae-woong explained this problem in terms of a flywheel, that is, a virtuous cycle structure. He pointed out that while the United States has turned the flywheel through long-term research on original technology and China through large-scale investment of funds, Korean policies often pursue individual projects in a segmented manner, making it difficult to see how the entire ecosystem is connected.

"If you're going to make a policy, you have to start by defining where the flywheel is."

The argument is that rather than a fragmented goal such as building a few thousand smart factories, we should design a structure in which when one field is ignited, the effect spreads to other fields and the entire end-to-end ecosystem grows together.

10. Safety and security in the age of agentic AI

Agentic AI goes beyond simply generating answers and performs actual actions such as making payments, executing code, changing systems, and accessing documents. As a result, much more complex security issues arise than existing generative AI.

Commissioner Lee Dong-soo emphasized that in an environment where multiple agents read and write the same memory, it is necessary to design from the beginning which agents can access which information. This is similar to the problem of managing access rights for each process or core in existing operating systems, but it can be more difficult because there are many more agents and they are complexly connected.

"Which agents can read which memories and which cannot, and this kind of permission structure needs to be built into the design from the beginning."

Although the number of cases where agentic AI is used in attacks may increase, defense must also utilize agentic AI. This is because it is difficult to handle automated attacks with human speed and patterns alone. Therefore, there is a need to establish inter-agent communication protocols and security standards at the national level.

Professor Lee Ki-min pointed out that current AI regulations mainly focus on the safety of generated output, such as harmful text, bias, and image display. However, in the era of agentic AI, the behavior itself, such as proxy payment, system access, and automatic execution, must be regulated.

Harness was presented as a key safety device. Harness is a control device that defines what authority is given to AI and to what extent it has access to which systems.

"Harness is a safeguard that defines what permissions a model can have, which systems, and how much access it can have."

Commissioner Shin Jeong-gyu explained that in a corporate environment, the same role-based access control (RBAC) as humans must be applied to agents. Just as you give people access to documents and data, you also need to give agents permission and manage their access records and actions.

However, in the future, when an environment uses multiple agents per person, the authority relationship between people and agents becomes very complex. If the ratio of people to agents exceeds 1 to 10, the authority structure can become exponentially more complex, so a new management system that goes beyond applying the existing RBAC is needed.

11. Education in the AI era must teach tool utilization and fundamental competencies together.

The final topic of discussion was education in the AI era. Since the current knowledge transfer and problem-solving-centered education are areas where AI is best at, it was discussed whether education should focus only on how to use AI or whether it should develop judgment and verification abilities that do not depend on AI.

Professor Lee Ki-min said that AI can quickly summarize papers and write sentences, but if students skip the process of reading the paper themselves and considering the structure, they may not be able to learn the ability to write papers properly.

"There are many things we can do quickly with AI, but there are some things we can only learn by doing."

The same goes for coding. In the future, AI may be better at coding than humans, but students can learn logical and structural thinking in the process of learning coding themselves. Just as the presence of a calculator does not eliminate the need to study math, the argument is that fundamental learning should not disappear just because an AI tool exists.

Vice President Lee Yun-soo emphasized that we must not only maintain the ability to utilize AI well, but also maintain cognitive sovereignty to think for ourselves. This means that the entire human judgment and thinking process should not be outsourced just because AI is convenient.

"It is still more important to nurture people who can think fundamentally rather than who are good at using tools."

Commissioner Lee Dong-soo said that we need to distinguish between basic competencies that must be learned in school and practical skills that must be learned in the corporate field. Even if the AI ​​field changes rapidly, basic knowledge such as mathematics, science, probability/statistics, and linear algebra becomes more important.

This is because basic mathematical thinking is needed to verify the results generated by AI and determine what adjustments to make next.

"It is difficult to judge and adjust the results produced by agent AI without basic mathematics such as probability and statistics."

He also emphasized that language and communication skills such as reading, writing, and speaking are important. In the AI ​​era, the ability to define problems, communicate with others, and verify results becomes more important than how to use fragmented tools.

Conclusion: The core of Korea's AI strategy is connection and independence

The key point repeatedly emphasized in this seminar is that Agent AI is not a simple model competition but a change in the entire computing ecosystem. Moving away from the existing structure centered on GPU and HBM, memory, storage, communication, operating system, harness, security, and services must all be closely connected.

Korea has strengths in the memory and semiconductor fields, but it is difficult to reach the top three in AI if it only supplies parts. A strategy is needed to understand the service market, design hardware and software together, and create a national-level flywheel and ecosystem.

At the same time, a certain level of self-sufficiency must be secured from models to infrastructure to avoid complete dependence on specific countries or companies. Competitiveness in the era of agentic AI is likely to be determined not by the largest model, but by how safely, quickly, and efficiently actual tasks can be performed.

Lastly, there was a consensus that education should preserve human basic knowledge, logical thinking, judgment, verification ability, and reading, writing, and speaking skills along with the ability to use AI.

"We need to develop people who can think for themselves while using AI beyond those who are good at using AI."