[D] Looking for advice: Modelling a medicine-reminder agent that must decide “remind / wait / notify” under incomplete information[D]

[u/Senior_Disaster_7307]

Hi everyone,

I’m researching how to design an AI agent for a medicine-reminder system. The agent has to decide, at each relevant time, whether to:

• send a reminder,
• wait (do nothing for now), or
• notify another person (e.g. caregiver),

when it does not have complete information about the patient (has the dose already been taken? is the person nearby/attentive? are there adherence barriers? etc.).

I’m trying to frame this properly before diving into implementation. Right now I’m looking at it as a sequential decision problem under partial observability (POMDP / belief-state RL territory), but I’m not sure how far that framing is actually useful in practice for this kind of system.

I’d really appreciate any pointers on:

1. Is a POMDP / belief-state approach overkill here, or is it the right formalization? What simpler alternatives (contextual bandits, MDP with engineered features, rule-based + uncertainty thresholds, etc.) have people used successfully for similar “remind vs wait vs escalate” decisions?
2. Papers, open-source projects, or real systems that tackle medication adherence / context-aware reminders with uncertainty or incomplete observations.
3. Common practical pitfalls (reward design, observation noise, alert fatigue, safety/escalation logic, evaluation metrics) that aren’t obvious from the theory.
4. Any recommended starting points for someone new who wants to move from “I understand the concepts” to a small working prototype or simulation.

I’m mainly in research/preparation mode right now, so even high-level advice, key papers, or “here’s what I’d do differently” comments would be very helpful. Thanks!

[u/keonechong]

What is the original frame of reference?

Starting there and mapping by forced action and reference artifacts is how I would start.

For example on of the steps would require thinking about when the reminder happens (time and frequency and yes/no) but that needs a hard determination, reference a log, is frequency envelope correct, prove it’s correct, move to the next gate, fire reminder. Did reminder fire, prove reminder fired, log reminder.

It’s really about mapping jtd against your deterministic workflows. And having proper checks and evals to make sure the agent isn’t lying.

It WILL lie frequently. Maybe 20ish prevent of the time I catch my Claude opus agent in a blatant lie.

[u/Level_Conclusion_939]

It's classic Frame Problem
Maybe you can use model prior knowledge + some few-shots in your prompt to set TTL for the facts and add some cron jobs on the backend. I have used this approach for clinic ai-bot and it's seems enough (CSAT increase up to +15%)

POMDP is a good one if you have got a two-three states and trained model for this specific task. But it's easily becomes crazy as you may have a very large number of states (i don't know what you are modeling, assuming states are something like "taking pills" or "needs a surgery operation")

IMHO, LLM prior knowledge is the best one if you want generalistic approach

[u/Level_Conclusion_939 → u/Level_Conclusion_939]

Also you can add some "levels of uncertainty" for every time period of you TTL to flag facrs/actions as "needs attention" or "emergency"

[u/Honest-Finish3596]

Why are you using reinforcement learning with an LLM for something which is better handled by a calendar app?

In this case, you just want to give the reminder at regular intervals until they say they've taken it, not speculate on whether they may or may not have. An unpredictable medicine reminder app (which only gives reminders if the app guesses you didn't take it) would be the worst thing!

Even disregarding the fact that there is no way it can reliably guess whether the user has or has not taken the medicine. I turn my phone on at 10AM, based on what features is it supposed to guess whether it should or should not tell me about my medicine scheduled at 8AM? The best thing to do is what every bog standard calendar app does which is just to issue the reminder.

[u/AlexanderDoak]

What is the total context size for all input information? Are the inputs purely unstructured? Purely structured? Or a mix of both? Sounds at first like a very straightforward three class classification problem. How easy is it for you to get gold-standard labeled examples (inputs with the correct label)? How many can you get? You might just be able to fine tune an LLM if you have enough data. You could also treat it like three separate binary classification problems so you can use the very clean precision, recall, F1, MCC family of metrics (and/or ROC AUC) to really optimize fully. But it all depends.