Hospital Price Transparency

Effective on 2021-01-01

Source: https://www.cms.gov/hospital-price-transparency

Civil monetary penalty (CMP) notices issued by CMS

Civil monetary penalty (CMP) = 民事罰款

Source: https://www.cms.gov/hospital-price-transparency/enforcement-actions

Shop site developed by Taipei DXP SBP team

Patient can search Procedure Price based on Payer Name and Medical Network

Source: https://shop.baconcountyhospital.com/

Machine Readable Files (MRFs)

Hospitals need to provide their link to MRFs

Source: https://shop.baconcountyhospital.com/pt-machinereadable.html

[ WHY ] Will AI Chatbot be the "new" UI?

Call Center

Did you call Help Desk before?

Image Source: "Telework Guy",by j4p4n, OpenClipArt, 2020-06-05

Slack Chatbot

Source: "Niles is a Slack bot that learns your team’s questions
and answers them so you don’t have to", 2017-03-17

ChatOps

Source: "Three ChatOps examples demonstrate DevOps efficiency", 2018-02-05

LINE Chatbot

Source: "LINE Bot Designer"

ChatGPT

NLP (Natural language processing)

Source: "How ChatGPT Works: The Model Behind The Bot", 2023-01-31

[ WHY ] Multilingual Support?

[ Pain ] Complex Architecture

( Note: I removed confidential diagram shown here )

Source: Reverse Engineering PlantUML diagram by Jazz Wang

[ Gain ] Needs to support multiple langauges

ChatGPT supports 95 languages

Source: "How to Use ChatGPT in Other Languages", 2023-05-25

Benefits Of Multilingual ChatGPT

Source: "List of languages supported by ChatGPT",
Botpress Community, 2023-03-23

[ HOW ] High-level Architecture

[ HOW ] High-level Architecture

[ HOW ] POC Implementation

.
├── LangChain-LocalAI
│   ├── docker-compose.yaml
│   ├── models
│   │   ├── gpt-3.5-turbo.yaml
│   │   ├── gpt4all-chat.tmpl
│   │   └── gpt4all-completion.tmpl
│   ├── requirements.txt
│   ├── titanic.db
│   └── titanic.py

└── LangChain-OpenAI                    ## Today's demo
    ├── Makefile                        ## simplify the step to run
    ├── README.md
    ├── bacon.py
    ├── chainlit.md                     ## Chainlit README page
    ├── chainlit_sample.py
    ├── gen-faiss-index-from-mrf.py
    ├── gen-sqlite-from-mrf.sh          ## create SQLite as data source
    ├── hpt-chat.py                     ## Main Program
    ├── import.sql                      ## used by gen-sqlite-from-mrf.sh
    ├── lanarky_test.py
    ├── requirements.txt                ## required Python packages
    └── templates
        └── index.html

5 directories, 19 files

[Deep Dive] OpenAI API

Most poeple use ChatGPT (free) / ChatGPT Plus (paid).
By subscribing OpenAI API, you can create application programatically.

Source: https://platform.openai.com/apps

[Deep Dive] OpenAI API Capability

https://platform.openai.com/docs/quickstart?context=python

[Deep Dive] OpenAI API Keys

https://platform.openai.com/api-keys

[Deep Dive] OpenAI API Usage

https://platform.openai.com/usage

[Deep Dive] OpenAI Billing Auto Recharge

When you don't want to do experiment for a long term,
you can Cancel Plan to reduce cost.

https://platform.openai.com/account/billing/overview

[Deep Dive] LangChain - Introduction

LangChain is a framework for developing applications
powered by language models.

• LangChain enables applications that are:
  • Data-aware: connect a language model to other sources of data
  • Agentic: allow a language model to interact with its environment
• The main value props of LangChain are:
  • Components: abstractions for working with language models.
  • Off-the-shelf chains: components for accomplishing specific higher-level tasks

Source: LangChain Document - Introduction

[Deep Dive] LangChain - Modules

• Model I/O : Interface with language models
• Data connection : Interface with application-specific data
• Chains : Construct sequences of calls
• Agents : Let chains choose which tools to use given high-level directives
• Memory : Persist application state between runs of a chain
• Callbacks : Log and stream intermediate steps of any chain

Source: LangChain Document - Introduction

[Deep Dive] LangChain - Model I/O

• Prompts: Templatize, dynamically select, and manage model inputs
• Language models: Make calls to language models through common interfaces
• Output parsers: Extract information from model outputs

Source: LangChain Document - Model I/O

[Deep Dive] LangChain - Data Connection

• Document loaders: Load documents from many different sources
• Document transformers: Split, convert into Q&A format, drop redundant documents
• Text embedding models: Turn unstructured text into vector
• Vector stores: Store and search over embedded data
• Retrievers: Query your data

Source: LangChain Document - Data connection

[Deep Dive] LangChain - Chains

Chain = a sequence of calls to components, which can include other chains.

from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate

llm = OpenAI(temperature=0.9)
prompt = PromptTemplate(
    input_variables=["product"],
    template="What is a good name for a company that makes {product}?",
)

Source: LangChain Document - Chains

[Deep Dive] LangChain - Memory

By default, Chains and Agents are stateless

LangChain provides memory components in two forms:

• helper utilities for managing and manipulating previous chat messages.
• ways to incorporate these utilities into chains.

Source: LangChain Document - Memory

[Deep Dive] LangChain - Agents

The core idea of agents is to use an LLM to choose a sequence of actions to take.

In chains, a sequence of actions is hardcoded (in code).
In agents, a language model is used as a reasoning engine.

Source: LangChain Document - Agents

• Agent Types:
  • Zero-shot ReAct <-- We use this for POC
  • Structured input ReAct
  • OpenAI Functions
  • Conversational
  • Self ask with search
  • ReAct document store

Source: LangChain Document - Agent types

[Deep Dive] Chainlit

Chainlit is an open-source Python package
to build production ready Conversational AI.

Key features

• Build fast: Integrate seamlessly with an existing code base or start from scratch in minutes
• Copilot: Embed your chainlit app as a Software Copilot
• Data persistence: Collect, monitor and analyze data from your users
• Visualize multi-steps reasoning: Understand the intermediary steps that produced an output at a glance
• Iterate on prompts: Deep dive into prompts in the Prompt Playground to understand where things went wrong and iterate

https://docs.chainlit.io/get-started/overview

[Deep Dive] Chainlit integration with LangChain

There are reference example code to integrate Chainlit with LangChain

https://docs.chainlit.io/integrations/langchain

[Learnt] Limitations of LangChain and LocalAI

Lesson #1: Prompt Observability

with Chainlit, you can see the prompt created by LangChain
and the response from OpenAI GPT-3.5/4 LLM

Lesson #2: GPT4All-J can't code like GPT-3.5-Turbo

Question: which is better? Multimodal LLM vs Code LLM?

• TODO: explore Meta's Code Llama-2 LLM

Source: https://huggingface.co/codellama

Lesson #3: Impact of Database Schema

What if LLM can't understand the meaning of Schema?

[Future] Serverelss Architecture

• Serverless Architecture: Use AWS Lambda instead of AWS EC2
  • reduce operation cost

Source: How to Create a FREE Custom Domain Name for Your Lambda URL - A Step by Step Tutorial

[Future] Different RAG implimentation

Retrieval-Augmented Generation (RAG)

Source: RAG Cheatsheet V2.0 🚀, Steve Nouri, 2024-02-18

[Future] Interacting with APIs with LangChain

• Common Enterprise Software InfoSec Pratice:
  Application (backend API) subnet is different from database subnet.
  (Data Security)
• You can share OpenAPI (Swagger contract) with LangChain.
  It helps to reduce internal data leaking. (Data Privacy)

sharing Interface instead of Data

Source: https://python.langchain.com/docs/use_cases/apis