Handling customer EHR and portal credentials safely
A customer types their practice management password into your onboarding form. Now you hold a credential that opens every patient record at that practice. If you lose it, the practice reports a breach. If the password changes, your sync fails at 2 a.m. This post covers how to collect the credential without seeing it, how to store it, what to do when it stops working, and what to write to your logs.
Two kinds of credential
EHR APIs certified to 45 CFR 170.315(g)(10) use OAuth 2.0. The customer authorizes your app on the vendor's own login page, and your product receives tokens. The rule requires the authorization server to issue a refresh token valid for no less than three months to confidential apps, and to grant access on later connections without re-authentication when a valid refresh token is supplied. Your product never sees a password.
Payer portals and many practice management systems have no OAuth. There the customer gives you a username, a password and often a second factor. The rest of this post is about that case, and about the client secrets vendors issue to you.
A hosted connect form
Do not let the password pass through your application servers. Run a separate credential service with its own domain, deployment, access list and audit log. Your product opens the connect form from that service in a frame or a redirect. The form posts the password to the credential service only. The product receives a connection ID and nothing else.
POST https://connect.example.com/v1/connections
{
"customer_id": "cus_8d1f",
"system": "availity_essentials",
"username": "frontdesk@clinic.example",
"password": "...",
"totp_secret": "JBSWY3DPEHPK3PXP"
}
201 Created
{ "connection_id": "con_2k9x", "status": "pending_verification" }When your sync workers need a session, they ask the credential service to log in on their behalf and return a session cookie or a short-lived token. The workers never hold the password.
Encryption at rest
You cannot hash a portal password. You must send it back to the portal, so you store it reversibly. The NIST rules in SP 800-63B for salting and hashing passwords apply to your own users' passwords. Credentials you hold for a third party fall outside them. The same document recommends a reauthentication limit of 24 hours and an inactivity limit of 1 hour at AAL2. Use those as the ceiling for sessions your credential service issues to workers.
Use envelope encryption. A data key encrypts each credential, and a key in a key management service encrypts the data key. OWASP's secrets management guidance says not to store keys next to the secrets they encrypt unless those keys are themselves encrypted, and to audit who requested a secret, for which system and in which role.
The HIPAA Security Rule at 45 CFR 164.312 lists encryption and decryption as an addressable implementation specification, audit controls as a standard, and unique user identification as a required specification. A credential service that encrypts every record, logs every access and ties every access to one named service account meets those three points in one design.
Vendor client secrets
Treat your own API credentials the same way. athenahealth's onboarding guide states that a client secret cannot be viewed again after creation without rotating to a new one, and that rotation invalidates the previous secret. Store the secret in the credential service at the moment you receive it, and rotate it from there so the old and new values never live in a chat log or a ticket.
Re-authentication
A stored credential stops working for four common reasons. The customer changed the password. The portal enforced a second factor the connection does not have. The account locked after wrong codes. The account was deactivated for inactivity.
Each has a signal. Availity requires an authenticator app or hard token for multi-factor authentication and locks the account after too many wrong codes. UnitedHealthcare deactivates portal accounts after 90 consecutive days of inactivity and requires each user to have their own One Healthcare ID. Alabama Medicaid requires a time-based one-time password app on its provider portal.
Map each signal to a connection state and a customer action:
| Signal | State | Customer action |
|---|---|---|
| Login form returns with an invalid credential message | needs_password | Re-enter password in the connect form |
| New second-factor enrollment page appears | needs_mfa_enrollment | Enroll a TOTP device through the connect form |
| Lockout message appears | locked | Clear the lockout with the portal, then retry |
| Deactivation message appears | deactivated | Reactivate with the portal administrator |
Do not retry a failed login in a loop. A second wrong attempt moves the account toward a lockout that a person must clear. One failure moves the connection to the matching state and stops the workers.
Marking a connection down
A connection has one status field that every worker reads before it starts. When the credential service sees a terminal login failure, it sets the status, records the time and the reason code, and emits an event. The product shows the customer the state and the one action that clears it. Queued jobs for that connection wait rather than fail. When the customer completes the action, the credential service verifies the new credential with one real login before it sets the status back to active.
What to log and what never to log
OWASP's logging guidance lists data that must never appear in logs: authentication passwords, session identification values, access tokens, encryption keys and database connection strings. Add TOTP secrets and one-time codes to that list. A portal page captured for debugging must be stored with form values and cookies removed.
The same guidance lists events to log: authentication successes and failures, authorization failures and session management failures. For a credential service, log the connection ID, the system, the worker identity, the state transition, the reason code and the time. Log the fact that a password was read, by which worker, for which job. Never log the password length or any part of it.
When you apply these rules, a leaked log file tells an attacker that connection con_2k9x failed to log in at 02:14, and nothing else.