HiMe is a personal health agent platform that analyzes data such as heart rate, blood oxygen, sleep, and exercise collected from smartwatches and smartphones in real time within the user's hardware. To solve the uniformity and privacy issues of existing wearable analysis, we used a database as a key component and combined real-time detection and long-term user modeling. The evaluation showed that the robust local open source model was able to compete with some of the state-of-the-art hosted models, but the ability to reliably continue multiple conversations and interpret subjective health status from wearable data remained challenges.

1. Research background and emergence of HiMe

Smartwatches and smartphones now continuously measure everyday health, even outside the hospital. Heart rate, blood oxygen, heart rate variability, sleep, and exercise records are accumulated on the device, creating a long-term health record for each individual over several months or years.

However, existing wearable analysis methods mainly rely on predetermined statistics or fixed indicators. This method does not sufficiently reflect the flow of signals that change over time or the user's personal preferences. Conversely, if you try to input all of the raw data directly into a large-scale language model (LLM), the amount of data will be so large that it will easily exceed the model's context length.

LLM agents can be an alternative to this problem. Instead of reading all the data at once, agents can use tools to search for the data they need, decide for themselves what to analyze, and then explain the results in natural language.

"Agents can access health data through tools, decide what is worth analyzing, and explain their findings in natural language."

Existing research has largely proceeded in two directions. Clinical multi-agent systems target doctors and medical staff as their main users, and deal with event-oriented data such as medical records rather than patients' continuous wearable data. Personal coaching systems focus on exercise advice or journaling, but do not provide long-term management of a continuous stream of physiological data coming in in real time. Additionally, while typical self-hosted personal assistants are great for conversations and scheduling tasks, they are not designed to handle high-frequency biometric signals and months-long personal models.

To fill this gap, researchers proposed HiMe (Health Intelligence Management Engine). HiMe is an open source platform that supports continuous health monitoring and personalized insight generation by processing data coming from various wearable devices within the user's device.

HiMe's design is based on three principles:

1. Treat the database as a core component.
   Not only wearable signals, but also user profiles, personal preferences, agent memories, reports, work schedules, etc. are managed together in one database.

2. Optimize effectiveness and efficiency simultaneously.
   Instead of blindly using the largest model, find the right balance between quality, cost, and speed of response. Cheap statistical filters are always run, and complex LLM analyzes are invoked only when meaningful changes are detected.

3. Combines real-time processing and long-term user modeling.
   It allows you to identify both abnormal signs that occur momentarily and lifestyle patterns that continue for several months.

Researchers view personal health data not as a simple resource to answer one-time questions, but as a long-term asset that helps users become healthier over time.

2. HiMe's overall structure and data flow

HiMe consists of agent server, smartphone app, and wearable app. Rather than the user directly querying the database, an agent acts as an intermediary between the user and the data store.

Wearable devices measure signals such as heart rate, blood oxygen, heart rate variability, sleep, and exercise in real time. The measured values ​​are temporarily stored in a certain bundle on the device and then transmitted to the agent server via the smartphone. The server's data adapter unifies the different data formats for each device and organizes them so that the same measurement values ​​are not stored redundantly.

Organized data is stored in an integrated database for each user. This database contains the following information:

• Biological signals collected from wearables
• User profile and goals
• Memories accumulated by the agent
• Generated health reports
• Scheduled tasks
• Real-time monitoring rules
• Custom tables created by agents as needed

Users do not manipulate the database directly. Instead, the agent reads the data, analyzes it in multiple steps, and saves the results as reports, memories, or new tasks. Because public wearable datasets can also be played through the same adapter, HiMe can be used as both a real service platform and a health agent evaluation environment.

HiMe's agent server processes three types of events in one execution loop.

1. Message sent by user
2. Alerts triggered by real-time monitoring rules
3. Pre-scheduled tasks

When this event occurs, the interaction agent wakes up and, as needed, calls specialized subagents such as analysis agent, management agent, and planning agent. The agent reads the necessary information from the database, records the results back in the database, and returns a response to the channel through which the user sent the request.

In particular, the numbers in all reports are compared to actual database queries or code execution results. Therefore, you can check whether the agent is making up numbers without any basis, and users can check what data the results came from through the View Evidence function included in the report.

3. Four-layer design

Data Layer

HiMe stores information held by agents in one integrated database. Wearable signals, user profiles, and long-term memory can be analyzed together instead of scattered across separate repositories.

For example, when a user requests a health check report, the agent does the following:

1. Read the user's existing baseline and personal goals from the database.
2. Identify information related to the required analytical techniques.
3. Query the wearable signal table in several steps.
4. Write the analysis results as a natural language report.
5. Save the report to the database.
6. If necessary, add follow-up tasks to run the next morning.

Since each agent is granted unique table access rights, all operations can be safely restricted. HiMe does not fix specific health analysis flows one by one with code. It provides only basic tools and database structures, and is designed to allow agents to utilize the tables and tasks needed according to the situation.

Thanks to this structure, there is no need to create a separate agent each time a new feature is added. The required functionality can be extended to new tables or data structures.

Tool hierarchy

Because the database is central, the tool layer is relatively simple. Basically, what the agent does is read and write to the integrated database. Code execution tools are added here, and writing answers, editing files, creating dashboards, etc. can be extended on top of the same abstraction.

The code execution tool maintains the state created from previous calls, like a Jupyter notebook. Therefore, the agent can continue to utilize previous analysis results without having to rewrite the code from scratch every time.

Additionally, all messages sent externally go through a fact verifier. The verifier checks the record of tools used by the agent and checks whether the numbers included in the answer match the actual query results. Evidence of the verification process is stored in hash form, allowing the results to be tracked later.

Agent Hierarchy

The interaction agent that talks directly with the user is at the center. This agent calls the following specialized subagents when needed:

• Analysis Agent: Analyzes wearable data and answers health questions.
• Management Agent: Manages memory, user profiles, and recurring tasks.
• Planning Agent: Creates a health plan and monitoring schedule tailored to the user's goals.

Prompts are divided into modules such as system instructions, roles, personalities, responsibilities, and experiences. This method allows multiple agents to reuse a common structure and helps to efficiently utilize the model's KV cache by keeping the prompt prefix stable.

HiMe supports 15 LLM API providers, but to protect privacy, the researchers recommend running a local open source model using vLLM.

Interaction Layer

Users can access agents through messaging platforms such as WeChat and Telegram, as well as HiMe's built-in chat. The gateway registry remembers the channel through which the user sent the message and returns a reply through the same channel.

As conversations accumulate, the agent updates the user's memory and profile and sets active tasks based on the user's status. A task may be a schedule-based task that runs at a specific time, or it may be an event-based task that runs when a specific health event occurs. Result notifications are delivered to smartphones and wearables.

HiMe also includes a pixel art cat digital twin that provides a visual representation of the user's state. A cat's movements and facial expressions express its real-time health status in a simple and familiar way. 🐈

4. An approach that combines real-time detection and long-term modeling

An important feature of HiMe is that it does not call expensive LLM for every measurement. Because wearable data comes in so frequently, running complex analyzes on each heart rate measurement significantly increases costs and latency.

Instead, HiMe uses the following two-step structure:

1. Low-cost statistical triggers continuously monitor the entire stream.
2. When significant changes are detected, expensive LLM analysis is invoked.

For example, a statistical trigger may be activated when the heart rate becomes abnormally higher or lower than the personal baseline, blood oxygen falls below a certain level, or heart rate variability drops sharply. Only then does the analysis agent wake up and look at the data from the surrounding time zone and past records.

This structure simultaneously handles momentary anomalies and long-term trends. Inexpensive components continuously monitor data at high resolution, and LLM only steps in to provide interpretation and explanation when critical events occur.

The researchers explain that this design offers the following advantages:

"Transform a continuous physiological stream into personalized, auditable insights, while ensuring the data never leaves the user's hardware."

Additionally, the user's long-term model is not managed as an opaque vector database, but as a Markdown-based user model that can be directly modified by the agent. Users can view and modify memories, profiles, tasks, reports, prompts and tools directly from the server panel.

5. Actual user experience and demonstration process

HiMe's usage process continues from initial setup to daily health care. First, the setup wizard configures the LLM backend, time zone, messaging integration, and agent server. Afterwards, you approve access to health data on your smartphone, connect to the server, proceed with the consent process, and answer a simple health goal questionnaire.

The planning agent generates the following based on the user's survey answers and already collected health data.

• Personalized monitoring tasks
• Schedule regular health analysis
• Initial health planning
• Generating reports for sleep, activity, recovery, etc.

During daily use, the digital twin cat provides a summary of its health status. Users can request immediate analysis via chat when needed, and can also interact with agents through smartphone notifications, messaging apps, and wearable screens.

Reports generated by agents provide evidence based on database queries. Users can view collected data, agent status, reports, scheduled tasks, and personalization pages in the Advanced Management tab. The Agent Server panel provides detailed control over models, execution processes, prompts, memories, techniques, tools, and generated content.

HiMe therefore allows agents to perform health management autonomously, without leaving the operating process completely hidden from the user.

"Users benefit from autonomous health support, while retaining full visibility and control over how the agent operates."

6. Evaluation method and experiment configuration

Although HiMe is a demonstration system rather than a complete benchmark, it is designed to allow you to evaluate the agent's results against the final state of the database.

The researchers made the agent work by playing recordings contained in a public wearable dataset. The final database state created by the agent was then compared with the correct answer state independently calculated using the same data.

The following five datasets were used for evaluation.

-LifeSnaps -PMData -MMASH -CovIdentify

• GLOBEM

We used records from randomly selected participants in each dataset, and each dataset went through a separate data adapter. In total, 22 LLM backbones were compared, 19 of which were local models. Model sizes ranged from 1.5B to 35B parameters, and hosted API models such as GPT-5-mini, DeepSeek-V4-flash, and Gemini-3.5-flash were also included for comparison.

Evaluation items were divided by role.

• Routing: Does the interaction agent accurately call only the necessary subagents?
• Analysis: Does the natural language question present numbers that match independent standard queries?
• Memory Management: Modify the user's profile and active actions as requested
• Plan Creation: Create a schedule and report that reflects all of the user's health goals
• Illusion: How many times do we say numbers without any real basis for querying or executing code?

Each role has independent evaluation criteria, so when a failure occurs, you can determine which role caused the problem.

7. Agent performance by role

The evaluation results showed that robust local models can perform similar or better than some hosted models.

Typically, Qwen3.6-27B and Qwen3.5-35B-A3B each recorded an analysis score of 0.91. This was higher than DeepSeek-V4-flash's 0.85 and GPT-5-mini's 0.76. However, Gemini-3.5-flash showed the highest analysis score at 0.94.

"The strong local backbone reached an analysis score of 0.91, competing with the hosted model and causing no hallucinations."

However, performance did not always improve linearly as the model size increased. For example, Qwen3-30B-A3B is a mixed expert model whose total parameters are 30B, but the actual activated parameters are about 3B. This model scored lower in analysis and memory management than the smaller Qwen3-8B.

Additionally, there was a large difference in difficulty for each role. Routing was nearly saturated in the strong model, but analysis power varied greatly depending on the model, ranging from 0.00 to 0.94. In particular, the 1.5B class small model failed in most roles, and had the highest hallucination rate of 0.47, which refers to unfounded figures.

In the overall evaluation in the appendix, Qwen3.5-9B was introduced as a relatively inexpensive, practical local configuration with an analysis score of 0.87 and a memory and planning score of 1.00. However, showing good performance in individual tasks and performing multiple conversations reliably were two different issues.

8. Performance drops dramatically in multi-step conversations

If you only look at the performance of individual roles, the model may appear competent enough. However, in reality, users don't just ask one question, they have multiple conversations to make plans, remember previous information, apply new units, or check past records.

Reflecting this, the researchers conducted a complex conversation evaluation consisting of the following five steps.

1. Create a weekly review plan that includes sleep and activity.
2. Remember the plan you just set.
3. Calculate your average heart rate on your most active day.
4. Store the user's persistent preferences in memory.
5. Change the units to reflect your previous preference and answer with your highest heart rate ever.

Although the average performance of individual conversation steps was high, the rate of completely passing all steps was low. This is because small errors that occur in one step accumulate to the next step.

The highest strict pass rates were only 0.44 for Gemma-4-26B-A4B and 0.37 for Gemini-3.5-flash. For many models, the rate of successfully completing all steps was 0.

"The ability to reliably connect multiple roles within one session is a key unresolved challenge in on-device agent research."

This shows that the ability to simply answer questions accurately is different from the ability to maintain context, memory, and tool use consistently as a long-term personal assistant.

9. Does separation of roles actually help?

Rather than having one large agent perform all tasks, HiMe divides roles into routing, analysis, memory, and planning. The researchers compared this decomposition with a single-agent structure to determine whether it was simply a software architectural choice or whether it also helped actual performance.

In most models, the role-divided structure maintained or even improved performance. In particular, the following improvements were observed:

• Qwen3-8B: approx. +0.65
• Qwen3-32B: approx. +0.23

Most of the 13 local models did not suffer significant losses due to role separation. However, Qwen3.6-27B and Gemma-4-E4B showed exceptions where performance was lower than that of a single structure. The smallest models performed poorly on both structures.

Separation of roles narrows the responsibilities of each agent, simplifies tool calls, and allows error sources to be traced by role. The researchers explain that in a real operating environment, a structure with separated responsibilities is likely to be more advantageous than a single agent performing all roles within one long context.

10. Ability to interpret wearable data into health reports

Retrieving and calculating numbers is one thing, but interpreting how those numbers relate to the user's fatigue, recovery state, and stress is another matter.

The researchers provided the agent with records of various vital signs of participants and had it create a personalized health report using only the wearable data. The results were compared with participants' separate self-reports to assess interpretations of fatigue, preparedness, and stress.

The model that received the highest score in this task was only 46%. Interestingly, Gemini-3.5-flash, which scored 0.94 in the single-question analysis score, only scored 14.6% in the report evaluation of interpreting subjective states from data. nemotron-3-nano, which had an analysis score of 0.85, also scored 18.7% in the report evaluation.

In other words, the ability to accurately pinpoint numbers does not mean the ability to properly understand and explain health conditions.

"The ability to describe subjective states is a separate competency from the ability to retrieve numbers."

These results show that additional research and evaluation criteria are needed for personal health agents to go beyond simple question-and-answer systems and interpret the user's actual condition and context.

11. Effect of real-time trigger method

Always-on health agents must simultaneously achieve continuous data monitoring and cost reduction. Instead of polling, which runs LLM analysis on all data at regular intervals, HiMe uses statistical triggers written by agents.

In the experiment, abnormal events such as heart rate spikes and drops, persistent tachycardia, low blood oxygen, and low heart rate variability were inserted into the data. Afterwards, the following two methods were compared.

• Polling method: Runs the same multi-step analysis every 5 minutes
• Trigger Method: Run analysis only when statistical conditions are met

The trigger method found more abnormal events on average than the polling method. Especially in small models, the advantage of the trigger method was greater because if too many normal signals were mixed, analysis running on a fixed schedule could easily miss important events.

In large models, there was a cost of slightly lowering the detection rate, but the effect was to reduce token usage by 1/10. Therefore, the trigger method was evaluated as a realistic method to reduce the cost of LLM calls without significantly damaging the detection ability of real-time monitoring.

12. Two months of real user research

To determine whether HiMe helps real users manage their health, researchers conducted a two-month field study with nine participants with no technical background. After using HiMe on their devices, participants rated usability, activeness, personalization, and accuracy on a 5-point scale.

The overall evaluation was positive, ranging from 4.00 to 4.67 points. The highest scores were:

• Usability: 4.67 points
• Active support experience: 4.56 points

Participants especially evaluated that active reports on sleep and recovery allowed them to re-check health data that was usually overlooked. The biggest advantage of HiMe was that it improved activity and personalization compared to existing tools.

On the other hand, the suitability of the personalized plan was the lowest at 4.00 points. Some participants felt that when their lifestyle patterns changed, their scheduled health plans did not adapt quickly enough to the changes. There were relatively few evaluations of significant improvements in accuracy or response speed.

Although reported errors were rare and mostly minor, the study was limited by the small number of participants and the fact that participants were volunteers. So while these results show positive promise, long-term studies with larger and more diverse user populations are needed.

13. Appendix Experiment and Implementation Details

The appendix expands the main experiment and explains the evaluation procedure in detail.

Overall role evaluation

A full evaluation of the 22 models showed that the latest local models significantly narrowed the gap with the hosted API model. However, models that received high scores in a single task were not stable in long-term conversations. For example, nemotron-3-nano recorded an analysis score of 0.85, but the composite conversation score involving multiple steps dropped significantly to 0.02.

Role decomposition experiment

In the role decomposition experiment, 13 local models were compared with single-agent and multi-role agents. For most models, performance was maintained even when roles were divided, and some models showed significant improvements. On the other hand, exceptions were identified in some powerful models where performance was reduced due to role separation.

Trigger detection protocol

In the trigger experiment, the same agent was executed using polling method and trigger method, respectively. The agent directly created detection rules including signals, thresholds, observation times, and replay timeouts based on the user's natural language requests. Adverse events included sudden increases and decreases in heart rate, sustained increases in heart rate, an 86% drop in blood oxygen, and low heart rate variability.

A fixed LLM evaluator determined whether the analysis actually detected an abnormal event, and cases where issues were raised without basis were treated as false alarms.

On-device latency

The local model was run on NVIDIA H200 with vLLM and bfloat16 to measure latency. The 32B model generated approximately 53 tokens per second, and the 1.5B model generated approximately 536 tokens. The speed of the mixed expert model was determined by the number of parameters actually activated rather than the total number of parameters, and the 30B class mixed expert model also operated at a similar speed to the dense 4B model.

User Survey

Participants initially used HiMe primarily as a sleep management tool, but later expanded its use to include exercise and stress management. Before using HiMe, health information was often distributed across multiple apps. Participants evaluated that proactiveness and personalization were improved compared to existing methods, but relatively less emphasis was placed on improvements in accuracy, speed, and usability.

14. Research conclusion and remaining tasks

HiMe is a self-hosted platform that processes health data continuously flowing in from wearable devices within the user's device and turns it into personalized long-term health insights. The system was designed based on three principles: integrated database, co-optimization of quality, cost and speed, and combination of real-time sensing and long-term modeling.

Experiments show that a sufficiently powerful local model can compete with some hosted state-of-the-art models. This means that the possibility of building personal health agents without sending personal information to an external server is increasing. Additionally, the use of low-cost statistical triggers allowed for both continuous monitoring and reduced LLM call costs.

However, it is clear that there are still problems that need to be solved.

• Ability to maintain memory and tasks reliably across multiple conversations
• Ability to interpret subjective states such as fatigue, stress, and recovery from wearable measurements
• Ability to quickly adjust health plans to the user's changing lifestyle patterns
• Low accuracy and high hallucination rate of small on-device models
• Verification through more participants and long-term actual use

The researchers hope HiMe will serve as a foundation for researching and evaluating trustworthy and personalized health agents.

"Robust local models can already compete with hosted APIs, but multi-turn reliability and data-to-report interpretability are still open challenges."