Unsupported Browser
The American College of Surgeons website is not compatible with Internet Explorer 11, IE 11. For the best experience please update your browser.
Menu
Become a member and receive career-enhancing benefits

Our top priority is providing value to members. Your Member Services team is here to ensure you maximize your ACS member benefits, participate in College activities, and engage with your ACS colleagues. It's all here.

Become a Member
Become a member and receive career-enhancing benefits

Our top priority is providing value to members. Your Member Services team is here to ensure you maximize your ACS member benefits, participate in College activities, and engage with your ACS colleagues. It's all here.

Become a Member
ACS
Case Study

Performance Improvement in a Hip Fracture ERAS Program

Good Samaratin Medical Center

General Information

Institution Name: Good Samaritan Medical Center

Primary Author and Title: Christina L. Henderson, MS

Co-Authors and Titles: Thomas T. Mydler, MD; Barbara Stewart, MSN, RN, CPN; Rebecca C. Davis, RN, BSN; Dana Nordquist, BSN, RN; Kim Tate, PharmD, BCPS, MHA; and John H. Eisenach, MD

Name of Case Study: Performance Improvement in a Hip Fracture ERAS Program

What Was Done?

Global Problem Addressed

Hip fractures are a major public health concern in the United States with more than 100,000 hip fracture repairs performed annually.1 A large proportion of hip fractures occur in the geriatric population and are often presented with comorbidities.2

The consequences of a hip fracture consist of a one-year mortality rate of more than 30 percent, functional loss, and estimated medical expenses of more than $40,000.3 With an aging population, the incidence of hip fractures will continue to increase. Although there has been an improvement in surgical interventions, the rates of subsequent fractures are high at 30 percent for women and 22 percent for men.4 Of the subsequent fractures, the mortality rates for men and women are 74 percent and 49 percent, respectively.4 One method to improve the outcomes of hip fracture patients is the adoption of an Enhanced Recovery After Surgery (ERAS) program. ERAS is a multidisciplinary approach to perioperative care with the goal of standardizing patient care to reduce the physiological stress of surgery, complications, and mortality. By minimizing the variation in perioperative care and implementing a patient-centered team approach that coordinates all phases of care, the value of care (quality ÷ cost) should improve.5

Identification of Local Problem

Previous semi-annual reports from the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP®) presented an opportunity for improvement in the care of our hip fracture patients. Namely, risk-adjusted rates of postoperative mortality, delirium, venous thromboembolism (VTE), and surgical site infection (SSI) were consistently in the upper deciles.

Therefore, in 2018, we joined the Agency for Healthcare Research and Quality Safety Program for Improving Surgical Care and Recovery (ISCR) with an emphasis on hip fractures. We focused on patients undergoing the following hip surgeries: open reduction, internal fixation, hemiarthroplasty, and/or total arthroplasty. Our overall goal was to standardize and integrate all phases of care. Our specific aims were to improve postoperative mortality, delirium, SSI, urinary tract infections (UTI), VTE and length of stay (LOS).

How Was the Quality Improvement (QI) Activity Put in Place?

Context of the QI Activity

Good Samaritan Medical Center is a community based, 250-bed level II trauma center and is part of the SCL Health integrated network. As a long- standing member of ACS NSQIP, we formed an ERAS task force to develop and implement ERAS service lines for elective colorectal surgery in 2016, and elective total hip and knee replacement in 2017. These initiatives produced substantial health care value and provided the motivation to implement a care pathway for hip fracture patients, which coincided with the ISCR program’s development of a hip fracture ERAS pathway in 2018. Our hospital admits approximately 200 patients with hip fractures per year.

Planning and Development Process

The ERAS task force directed its efforts to hip fractures in 2018. The urgent, non- elective nature of patients admitted for hip fractures, particularly during “off-hours,” required expansion of our team to include the Emergency Department (ED).

We also recruited physician hospitalists and nursing leadership (Chief Nursing Officer, directors, managers) because some hip fracture patient admissions were transferred directly from outlying health facilities to the hospital floor, and thus bypassed the ED process. The task force was led by the ACS NSQIP physician champion who also serves as the Medical Director of ERAS. The exploratory meeting with all stakeholders began with the question: “In your phase of care, what is an identifiable opportunity gap for improvement?” As every representative expressed key concerns and suggestions, it became obvious that an ERAS pathway for hip fracture was a critical need. Subsequent meetings were held monthly to build a pathway that synergized care within and among the care phases.

Description of the Quality Improvement Activity

The ACS NSQIP surgical clinical reviewer (SCR) served as the ERAS coordinator for hip fracture. This coordinator worked in conjunction with representatives in all phases of care. Because our hospital was the first to develop hip fracture ERAS in the SCL Health system, our Clinical Informatics (CI) specialists assisted in developing subsets of hip fracture ERAS-specific orders nested into pre-existing medical and surgical hospital order sets. One key addition was a dietary order that allowed solid food intake until 10 hours before the anticipated surgery start time, and clear liquids until 4 hours prior to the surgery start time. These stop times added 2 hours to the American Society of Anesthesiologists NPO guidelines to account for the possibility of surgeries being moved to earlier in the day. Another key addition was a delirium prevention protocol ordered for all patients admitted with a hip fracture.

The medical director of ERAS created and disseminated a detailed pathway from admission to post-discharge, and a “pocket guide” highlighting the key steps in each phase of care. These materials were developed from published literature, the ISCR hip fracture resource library, and our task force consensus. The director also presented at the Lunch and Learn, our hospital-wide grand rounds. The SCR/ ERAS coordinator led small-group meetings and provided education materials to representatives from each phase of care.

The physician director of the ED and the ED nursing leaders relayed the education materials to the ED department providers and staff. An informational handout was created to provide education to the patient and family members (Appendix 1).

The ED physicians received training on fascia iliaca compartment (FIC) blocks and adopted the rationale for allowing, at a minimum, water intake. Opioid-sparing multi-modal analgesics were encouraged in addition to the FIC block. The directors of hospital medicine emphasized the need for rapid medical clearance to expedite the time from admission to surgery,6 the importance of allowing solid food and clear liquid intake within ASA guidelines prior to the expected time of surgery, and principles of opioid-sparing analgesics.7 These action items provide a promising strategy to reduce the incidence of perioperative delirium.8 In total, we considered pain, dehydration, opioids, polypharmacy (i.e., benzodiazepines, anticholinergics), environmental factors, urinary retention, and constipation as major precipitating factors for delirium.9

A final element specific to patients admitted for hip fracture was a process checklist to be completed by providers in each phase of care (Appendix 2). This “shared note” was added to the EPIC medical record after an ED administrative assistant sent a text alert to the SCR/ERAS coordinator and the Medical Director of ERAS, notifying of a new admission. This was necessary because hip fracture admissions occurred at all hours and providers needed a “behavioral nudge” to institute the ERAS-specific orders and care processes, including the preoperative diet and delirium prevention strategies. The preoperative diet order was especially challenging for multiple reasons: (1) the anticipated surgery start time was subject to operating room availability and the time required for medical clearance of the patient; (2) although multiple diets were available in the EPIC ordering system (i.e., regular, diabetic, clear liquids, NPO), there was no order option accompanying the appropriate diet with stop times for solids and liquids; and (3) the EPIC ordering system is a different platform than what is utilized in the hospital kitchen. As such, an ordering provider could free-text instructions for solid and liquid stop times, but the kitchen would not see the free text. This means that the patient or the family could phone the kitchen ordering system and receive a meal delivery in discord with the EPIC physician order. An “NPO violation” was considered unacceptable as this could lead to surgical delays despite the effort to minimize time from admission to surgery. To address this barrier, we created a new physician order called “Nurse to Place NPO Order,” which allowed the floor nurse to discontinue the solid order and subsequent clear liquid order 10 and 4 hours prior to surgery, respectively.

Several “soft rollout” ERAS items were adopted in early 2019, such as FIC blocks in the ED and spinal anesthesia in the OR. The official launch with all aspects of the pathway, including the patient education handout, the ERAS checklist upon admission, and preoperative diet, was August 21, 2019. The traditional care approach and the ERAS core principles are listed in Table 1.

Table 1. Core Principles in Hip Fracture Care with ERAS Implementation

Core Principle 

Traditional Approach 

ERAS Implementation 

Patient/Family Education 

No written information given prior to ERAS 

ERAS information sheet given to patients and family members 

Diet

NPO after midnight 

Water allowed in ED 

Solids until midnight before surgery Clear liquids >4 hours before surgery 

Multimodal Pain Management 

Opioid-based analgesia

FIC block in ED 

Non-opioid multimodal analgesics 

Opioids for breakthrough pain, no patient-controlled analgesia (PCA) 

Delirium Screening and Prevention

Delirium prevention protocol not universally ordered 

Automatically assess if <65 yrs old and at-risk patients <65 yrs old 

ICDSC score 4: notification of hospital medicine 

Time from Admission to Surgery

Within 48 hours 

Within 30 hours 

Surgical Site Infection 

Optional bed bath with bath wipes prior to surgery 

“Nose to Toes” chlorhexidine gluconate wipes the night before surgery, repeated in preop unit 

Mobilization

Physical therapy begins the day after surgery (POD 1) 

Out of bed (OOB) day of surgery Physical therapy day of surgery 

Ambulate within 12 hrs of surgery 

FIC: fascia iliaca compartment; ICDSC: Intensive Care Delirium Screening Checklist

Resources Used and Skills Needed

Staff

The ERAS task force and number of participants is listed in Table 2.

Costs

There were no additional costs beyond normal hospital operations to implement and maintain the QI program.

Table 2. ERAS Task Force

Stakeholder 

Number 

ACS NSQIP/ERAS Physician Champion 
ACS NSQIP Surgical Clinical Reviewer / ERAS Coordinator 
Orthopaedic Surgeon 
2* 
Orthopaedic Surgery Physician Assistant 
2* 
Anesthesiologist 
2* 
Hospital Medicine Physician 
2* 
Emergency Department Physician Director 
Emergency Department Nursing Leadership 
Chief Medical Officer, sponsor of ERAS project 
Chief Nursing Officer 
Director of Clinical Pharmacy 
Trauma Nurse Specialist 
Hospital Floor Nursing Leadership 
Perioperative Nursing Leadership (preop unit, OR, postop unit) 
Infectious Disease Officer 
Dietary and Nutrition Specialist 
Orthopaedic Care Specialist / Navigator 
Physical and Occupational Therapy 
Clinical Informatics 
Total 
30 

* Indicates two personnel were required in each category, one representing a physician group that serves patients in a Health Maintenance Organization (Kaiser Permanente) and private practice orthopaedic and internal medicine physician groups that serve patients apart from the HMO.

What Were the Results?

Overall Results

The patient characteristics and process measures are listed in Table 3. We excluded patients receiving percutaneous hip pinning from analysis, because this surgery is minimally invasive and can be performed under local anesthesia and sedation. Our control group consisted of all hip fracture patients in 2018 to eliminate the confounding effects of “soft rollout” interventions that were developed in early 2019. The percentage of patients with preoperative dementia was greater in the pre-ERAS group, which may be attributed to a slightly older population. Alternatively, the emphasis of ERAS interventions (FIC blocks, opioid-sparing multimodal analgesics, delirium prevention protocols upon admission, clear liquid diet) may have reduced preoperative mental status changes and decreased the chance of listing dementia in the hospital problem list. The time from admission to surgery was improved in the ERAS group. The use of FIC blocks, regional anesthesia in the OR, and the use of tranexamic acid was greater in the ERAS group. The other process interventions that were novel in the ERAS pathway did increase from pre- ERAS, but there is certainly capacity for improvement in these categories.

Table 3. Patient Characteristics and Process Measures
Characteristic 
Pre-ERAS n = 194 

ERAS n = 140 

Date range of surgery 

01/01/2018 – 12/31/2018 

08/21/2019 – 03/31/2020 

Mean age (yr) 
81 
79 
Sex (F:M) 
134:60 
82:58 
Preoperative dementia 
13% 
5% 
Admission to surgery < 30 hrs 
80.8% 
83.8% 
Fascia iliaca compartment (FIC) block in ED 
16% 
40% 
Allowed clear liquids until 4 hours prior to surgery 
N/A 
43% 
CHG wipe in preop unit 
N/A 
50% 
Use of regional anesthesia 
27.9% 
52.1% 
Use of tranexamic acid in OR 
8% 
40% 
Out of bed on day of surgery 
N/A 
40% 
Postoperative scheduled Tylenol 
N/A 
49% 

Data for ERAS patients derived from ISCR reports from August 21, 2019, to March 31, 2020. 

As shown in Table 4 the major finding of our QI program is that the 30-day mortality median length of stay was reduced from 5 days to 4 days after implementation of ERAS. Following ERAS implementation, the rate of delirium was reduced from 32 percent to 19 percent (p < 0.05). Since delirium is the most common complication after surgery in the geriatric population, we were pleased to see a drastic reduction in delirium event rates.10 Postoperative VTE requiring therapy were reduced from 3 percent to 2 percent. Rates of postoperative UTIs unexpectedly increased from 3 percent to 4 percent, despite encouragement of urinary catheter removal on POD 1.

Table 4. Outcome Measures before and after Hip Fracture ERAS Implementation
Outcome 
Pre-ERAS n = 192 
ERAS n = 167 
Mortality (no.%) 
8% 
6% 
Length of stay (median), days 
5.0 
4.0 
Length of stay (mean ± SD), days 
5.15 
4.9 
Postoperative delirium 
32% 
19%* 
Blood transfusion 
1% 
3% 
Discharge to home instead of care facility 
16% 
21% 
Surgical site infection 
1% 
0% 
Urinary tract infection 
3% 
4% 
Venous thromboembolism 
3% 
2% 

Data for ERAS patients derived from ISCR reports from August 21, 2019, to March 31, 2020, and manual abstraction. Delirium events, blood transfusion events, and discharge placement were tracked during hospitalization. Mortality, surgical site infection, urinary tract infection, and venous thromboembolism were tracked for 30-days post-discharge. *p < 0.05.

Setbacks

Major barriers that we encountered during the QI activity implementation centered on the complexity of coordinating all personnel in every phase of care. Busy EDs and physician groups have many rotating on-call providers which evokes a natural variation in familiarity with the ERAS protocol. These challenges are exacerbated by limitations in the EPIC medical record system for off-hours admissions. For instance, ERAS programs for elective surgeries allow pre-hospital education, health optimization, standardized nutritional support, and carbohydrate fluid administration on the morning of surgery. In contrast, our hip fracture program relied upon around- the-clock notification from the ED administrative assistants via text messaging.

By remote access into EPIC, we manually entered a shared note checklist upon admission and asked representatives from each phase of care to contribute to the shared note. Completion of the shared note was also difficult due to the large volume of nursing personnel during each care phase. Direct communication was required to advise providers on the proper handling of preoperative diet orders, which was not always feasible. A final setback and limitation of this report was the inability to automatically extract opioid administration from the EPIC medical record, which would result in mean morphine equivalent data for each patient.

A solution to these barriers may be addressed by a “Care Pathway” system within EPIC, that conceivably would automatically activate the ERAS pathway for hip fracture patients who are admitted through the ED. The system would then guide providers along each step of the pathway. Moreover, improvements are needed for a simple ordering system to allow clear liquids until 4 hours before surgery, and an electronic method to abstract total opioid use.

We revised our original QI plan regarding the preoperative diet. The “Nurse to Place NPO Order” was an undue burden on the busy floor nurse. Although we did not encounter NPO violations, we soon realized that allowing solid foods until 10 hours before surgery was too cumbersome, and we simplified this strategy by stopping solid foods at midnight. Another opportunity gap we need to address

is ongoing vigilance of delirium risk. While the delirium prevention protocol is helpful, we have less ability to track whether the hospital physician was notified if an ICDSC score was increasing for a given patient, and what action items were instituted to ameliorate delirium.

Cost Savings

  • The approximate amount we invested was minimal, aside from meeting times and participation of hospital staff.
  • The financial analyst at our hospital system estimated that reducing LOS for hip fracture patients by one day saves $593 per patient. For the present ERAS cohort, this saved approximately $90,000. For a full year of approximately 200 admissions, the savings would be roughly $118,600.

Tips for Others

  • The key aspect of our launch was to keep a common and achievable goal in mind while establishing new protocols in the ED. After establishing our goal we focused on educating hospital staff and patients during the process to ensure we were collecting accurate and measurable data. For example, if a FIC block was performed in the ED the procedure notes were included in the ED notes along with any complications. Once we decided to measure percentage of patients receiving a FIC block it was easy to access the data and determine if the FIC block occurred in the ED or after hospital admission.
  • Meetings occurred monthly at Good Samaritan Medical Center and were key to meeting our deadlines. During meetings it was helpful to discuss our successes, setbacks, and necessary changes. We used a master excel spreadsheet to keep track of our data. Some data was pulled from ISCR reports and some was manually abstracted.
  • A leader is important during the process to ensure the team is keeping on track with the ERAS goals. It can be a complicated and tiring process to launch a new program; therefore, a key stakeholder is necessary to manage all of the milestones during the process.

References 

  1. Siletz A, Childers CP, Faltermeier C, et al. Surgical Technical Evidence Review of Hip Fracture Surgery Conducted for the AHRQ Safety Program for Improving Surgical Care and Recovery. Geriatric Orthopaedic Surgery & Rehabilitation. 2018;9:215145931876921. 
  2. Edelmuth SVCL, Sorio GN, Sprovieri FAA, Gali JC, Peron SF. Comorbidities, clinical intercurrences, and factors associated with mortality in elderly patients admitted for a hip fracture. Rev Bras Ortop. 2018;53(5):543-551. 
  3. Brauer CA, Coca-Perraillon M, Cutler DM, Rosen AB. Incidence and Mortality of Hip Fractures in the United States.Survey of Anesthesiology. 2010;54(3):113-114. 
  4. Bliuc D. Mortality Risk Associated With Low-Trauma Osteoporotic Fracture and Subsequent Fracture in Men and Women. JAMA. 2009;301(5):513. 
  5. Ljungqvist O, Scott M, Fearon KC. Enhanced Recovery After Surgery: A Review. JAMA Surg. 2017;152(3):292-298. 
  6. Klestil T, Röder C, Stotter C, et al. Impact of timing of surgery in elderly hip fracture patients: A systematic review and meta-analysis. Sci Rep. 2018;8(1):13933. 
  7. Liu VX, Rosas E, Hwang J, et al. Enhanced Recovery After Surgery Program Implementation in 2 Surgical Populations in an Integrated Health Care Delivery System. JAMA Surg. 2017;152(7):e171032. 
  8. Malik AT, Quatman CE, Phieffer LS, Ly TV, Khan SN. Incidence, risk factors and clinical impact of postoperative delirium following open reduction and internal fixation (ORIF) for hip fractures: An analysis of 7859 patients from the ACS-ACS NSQIP hip fracture procedure targeted database. Eur J Orthop Surg Traumatol. 2019;29(2):435-446. 
  9. Rosen T, Connors S, Clark S, et al. Assessment and Management of Delirium in Older Adults in the Emergency Department: Literature Review to Inform Development of a Novel Clinical Protocol. Adv Emerg Nurs J. 2015;37(3):183-E3. 
  10. Marcantonio ER. The Relationship of Postoperative Delirium With Psychoactive Medications. JAMA: The Journal of the American Medical Association. 1994;272(19):1518. 

 

Appendix 1. Patient Handout: Caring for Our Patients with a Broken Hip

Our Medical Center now has a care pathway for our patients with hip fracture, which is called Enhanced Recovery after Surgery (ERAS). This is a team approach that involves you, your loved ones, and our entire team of doctors, nurses, therapists, and support staff.

We have developed this care pathway because we know that when we all work together, from your hospital admission until your full recovery, we are giving you the best chance of having the best possible outcome.

Here is what you can expect during your hospital stay:

EMERGENCY DEPARTMENT

  • We will explain your diagnosis and give you explanations of what you can expect during your stay
  • You will be offered clear liquids to drink. If you are diabetic you will be allowed water. You will also receive IV fluids to avoid dehydration.
  • Our ED doctor will consider performing a nerve block procedure to reduce your pain.
  • We will treat your pain with a variety of medications to help reduce the amount of opioid narcotic medications that you are given.
  • By treating your pain with a variety of methods and allowing you to drink, we hope to reduce the chance of you having confusion.
  • You may be offered a urinary catheter to help you urinate until after surgery.
  • We will try to schedule your operation within 24 hours of arrival to the hospital.

HOSPITAL FLOOR BEFORE SURGERY

  • You will be allowed to drink clear liquids until 4 hours prior to surgery (diabetics get water only).
  • You will be allowed to eat up until midnight prior to surgery. Please do not eat or drink anything that is not provided by the hospital team.
  • We will continue to treat your pain with a variety of medications to help reduce the amount of opioid narcotic medications that you are given.
  • If you take certain blood thinners, you will receive treatments to improve blood clotting during surgery.

SURGERY

  • You will meet your surgeon and anesthesiologist for questions, answers, and consent for surgery.
  • You may be offered spinal anesthesia (medicine in your back, plus sedation) or general anesthesia (asleep with a breathing tube).
  • Water and other clear liquids may be offered in the recovery room.

HOSPITAL FLOOR AFTER SURGERY

  • We will get you out of bed for meals and start physical therapy as soon as possible.
  • If you have a urinary catheter it will be removed in a day or two.
  • We will treat your pain with a variety of medications to help reduce the amount of opioid narcotic medications that you are given.
  • You can expect to take blood thinners to prevent blood clots for a month or more after surgery.
  • We will coordinate your after-hospital care to ensure the safest and soonest recovery to get back to your way of life. Thank you.

Appendix 2. ERAS Hip Fracture Checklist “Shared Note” on EPIC

Welcome to the ERAS pathway! Our goals are to standardize care, optimize patient and family education, reduce pain and opioids, minimize delirium, allow appropriate food and fluids, operate within 24 hours of admission, encourage postoperative nutrition and ambulation, and optimize VTE prophylaxis and rehabilitation on discharge.

Ensure all items below are documented in the appropriate location in the chart.

ADMISSION/EMERGENCY DEPARTMENT

  • Diagnosed with hip fracture and added into EPIC List: {yes no}
  • Provide patient 1 page Talking Points education material: {yes no}
  • Nerve block in ED (fascia iliaca or femoral): {yes no} Date *** , Time ***
  • Foley catheter placement or external catheter at discretion of ED team: {yes no}
  • Water allowed until 4 hour before surgery start time; Non-diabetics may have clear liquids: {yes no}

INPATIENT UNIT-PREOP

  • Surgery performed within 24 hours: {yes no}
  • Provider places ‘Nurse to Place NPO Order’: {yes no}
  • Nurse places order/times for clear liquids AND NPO strict: {yes no}
  • Water ad lib until 4 hour before surgery; Non-diabetics may have clear liquids (i.e., apple juice): {yes no}
  • Solid foods up until 10 hour before surgery: {yes no}
  • Delirium Prevention Protocol: {yes no}
  • Provider notified of delirium screening score of ≥4: {YES/NO/NA}
  • Isogel bed: {yes no}
  • Begin sequential compression devices (SCD’s): {yes no}
  • Education materials (1 page Talking Points and Hip Fracture Booklet) reviewed with patient and family: {yes no}
  • Foley catheter placement or external catheter (if appropriate): {yes no}
  • CHG bath the night before or day of surgery: {yes no}
  • Receive scheduled non-opioid pain medication (example-Tylenol, NSAIDS or Gabapentin): {yes no}

SURGERY

  • Nose to toes in Pre-surgical Care Unit: {yes no}
  • Fascia Iliaca (FIC) block if no planned periarticular block by surgeon; may place FIC catheter: {yes no}
  • Spinal anesthesia (preferred, may be appropriate for INR up to 1.4 after r/b/a assessment): {yes no}
  • Tranexamic acid 10 mg/kg up to 1g given prior to incision: {yes no}
  • Periarticular injection by surgeon: {yes no}
  • PONV prophylaxis: {yes no}

INPATIENT UNIT POSTOP

  • Delirium Prevention Protocol: {yes no}
  • Provider notified of delirium screening score of ≥4: {YES/NO/NA}
  • VTE prophylaxis started within 24 hours of surgery, mechanical + pharmacologic:
  • {yes no}
  • Advance diet as tolerated, eat meals out of bed: {yes no}
  • Patient mobility: OOB POD 0--chair {yes no}, ambulate {yes no}
  • WBAT POD 1: {yes no}
  • Date of 1st ambulation (>10ft): {yes no} Date *** , Time ***
  • Foley removal within 24 hours after surgery: {yes no}
  • Receive scheduled non-opioid pain medication (example-Tylenol, NSAIDS or Gabapentin): {yes no}
  • Discharge planning initiated (Care management): {yes no}
  • Postoperative VTE prophylaxis plan (example-Lovenox x 4 weeks, ASA 81 BID x 4 weeks) ***