Supplemental Development Methods of BitDoctor AI Imaging Technology

Data Collection Apparatus The data collection apparatus consisted of a smartphone camera to record video for BitDoctor AI advance imaging lighting to evenly illuminate the subject’s face during video recording, a height-adjustable seat to position the subject in front of the camera at the appropriate height and distance, a table on which the subject could rest their arms during blood pressure measurement, and an FDA-cleared continuous blood pressure monitor to measure reference blood pressure.

An Apple iPhone 6 Plus smartphone (Apple Inc.) was used to record videos. The smartphone was mounted atop of a tripod at eye level and positioned with the front-facing camera facing the participant. The native camera application was configured to record video of the participant’s face at 30 frames per second and at a resolution of 720p. Two light emitting diode (LED) soft boxes evenly illuminated subjects’ faces during video recording. They were mounted atop tripods at face height and one was positioned on each side of the iPhone facing the subject. A height-adjustable chair was used to seat subjects approximately 40-60 cm in front of the camera and ensure that their feet could rest flat on the floor. A table was placed in front of the subject so that their forearms could rest on a flat surface with their upper arm at heart level.

We recorded the ambient room temperature throughout the year. Temperature at the testing center was maintained at 21 C – 22 C year-round by heating and air conditioning. Heating and air conditioning at the second testing center was applied more sparingly. During the winter months, heating helped maintain temperature at or above 17 C.

During the summer months air conditioning helped maintain temperature at or below 26 C. The spring season was devoid of indoor temperature control; temperatures during this time dipped as low as 14 C. The average temperature ± SD was 19 ± 4 C.

The AI Imagine technology predicts subjects’ arterial pressures from videos of their face. It consists of a computational algorithm that extracts blood hemoglobin signals, and a software-based processing tool that filters out noise in unwanted signal frequencies. The technology also includes an algorithm that extracts unique features from hemoglobin signal, and then two computational models that predict, respectively, a subject’s systolic, diastolic or pulse blood pressure based on these features. These are models trained using advanced machine learning techniques. The specifics of signal extraction, signal processing, and blood pressure prediction model training, and testing are described below.

An FDA-cleared continuous blood pressure monitor (CNAP® Monitor 500, CNSystems Medizintechnik AG, Graz, Austria) was used to measure reference blood pressure oscillations. This monitor continuously measures blood pressure through a finger cuff using the vascular unloading technique1,2, and a transfer function is applied to calculate upper arm blood pressure. The system is calibrated with a single discrete measurement from an automated upper arm blood pressure cuff. It provides clinically acceptable agreement with intra-arterial pressure measurement – the gold standard for continuous blood pressure measurement3,4. It produces a continuous beat-by-beat blood pressure trace so that we could compare the imaging waveforms with their counterpart blood pressure waveforms. This will be essential during signal extraction. The maxima of the oscillations correspond to systolic blood pressure and the minima correspond to diastolic blood pressure. This device continually measures blood pressure in the finger using a technique called vascular unloading. This technique applies a counter-pressure to the finger to keep arterial blood volume constant. Then, it detects hemoglobin concentration through optical means. Because arterial blood volume is held constant, hemoglobin concentration is proportional to blood pressure. It then converts finger measurements into upper arm blood pressure readings. The device determines this conversion factor by oscillometrically measuring the subject’s upper arm blood pressure during calibration. Blood pressure traces were saved via a data connection to a Data acquisition system.

Calibration Procedure for Continuous Blood Pressure Monitor To calibrate the CNAP® Monitor 500, experimenters fitted the subjects’ right index finger and middle finger with the CNAP® Monitor 500 finger cuffs and secured this device to the subject’s forearm with a Velcro strap. Either finger cuff may be used during the experiment. Experimenters then fitted the subject’s right upper arm with the CNAP® Monitor 500 calibration cuff. Subjects were then asked to adjust their arm position on the table so that the upper arm blood pressure cuff was at heart level. Experimenters then initiated device calibration. Calibration began with initialization of the finger cuff, followed by an automated oscillometric blood pressure measurement of the upper arm. This reading was used to determine a finger pressure-to-upper arm pressure conversion factor. This conversion factor was used to convert all subsequent finger pressure measurements, so that standard upper arm blood pressure readings would be obtained from continuous blood pressure measurement in the finger. This calibration process lasted approximately 3 minutes.

Signal processing (Reference blood pressure)

Blood pressure traces generated by the CNAP® Monitor 500 and acquired by AcqKnowledge software were exported to MATLAB software for analysis. An example blood pressure trace is depicted in Figure 3C (in red). The maxima (peaks) within this trace correspond to systolic blood pressure, and the minima correspond to diastolic blood pressure. The difference between the two corresponds to pulse pressure.

Signal processing (BitDoctor AI Technology) Each pixel within each frame of the video contains 8 bits, each of red, green and blue (RGB) colour information. We first separated information in each frame of the video by colour channel. Within each channel, the color intensity of red, green or blue at a given pixel is encoded in an 8-bit color stack, with a 1 or a 0 at each bit of the stack. We treated each bit of the stack as a layer, or bitplane. We proceeded to isolate layers of the stack where the 1’s fluctuated along with reference blood pressure using a machine learning-based computational model. We reasoned that these layers best indicated changes in hemoglobin concentration. The remaining layers were subtracted from the stack, thus separating oscillating hemoglobin signal from information unrelated to hemoglobin concentration5.

BitDoctor.ai used this data to create a full spatiotemporal map of hemoglobin concentration (example in Figure 3A) and ultimately settled on 17 regions of interest on the face (Figure 3B) with robust hemoglobin fluctuations. Subsequently extracted hemoglobin signal only from within these regions. Bitdoctor used a software-based face tracker from the Dlib open source software library to automate and standardize the identification of these 17 regions of interest across all subjects’ faces. The regions of interest (from 1 to 17) are: forehead (small), nose (bridge), nose, nose (tip), left cheek (narrow), right cheek (narrow), upper lip, lower lip, chin (small), right cheek (full), left cheek (full), chin (full), philtrum, nose tip, forehead (wide), left forehead and right forehead. They are depicted in Figure 3B.

Within each region, we averaged hemoglobin signal across all pixels in a given region of interest, resulting in a 2-minute hemoglobin signal for each of the 17 regions. The advantages of this were twofold. First, limiting signal extraction to regions having robust hemoglobin fluctuation maximized signal to noise ratio. Second, limiting the number of signals to 17 key regions facilitated processing and analysis by limiting computational demands. Subjects remained generally still throughout the recording.

Features extracted from subject data The first 126 of 155 blood pressure prediction features were extracted from facial blood flow signals from each participant’s 17 regions of interest. These features fall into the following categories: pulse amplitude, heart rate band pulse amplitude, pulse rate, pulse rate variability, pulse transit time (PTT), pulse shape, and pulse energy (Supplemental Figure 1). According to the existing literature, these features are known to contain signals correlated with blood pressure to varying extents6-9.

Blood flow signals contain features with established links to blood pressure. For instance, blood pressure changes can be inferred from the combination of heart/pulse rate (more blood pumped per minute/more cardiac output) and pulse transit time/pulse wave velocity (faster transmission of blood pressure pulse indicates stiffer arteries, thus equating to higher intra-arterial pressure)6,7,9. Heart rate variability is inversely correlated with activity of the sympathetic nervous system, which controls blood vessel diameter (and thus pressure) neurogenically8. The shape of pulse waves also contains some blood pressure information10.

Crucially, however, accurate prediction of blood pressure will require the addition of novel features that when combined through machine learning will yield added information about blood pressure and enhance prediction accuracy. Machine learning enables us to combine newer and more abstract features (e.g., frequency filtered signals) in an unsupervised way to help predict blood pressure.

BitDoctor AI imaging technology used pulse waveform amplitudes of the imaging signal as features because waveform amplitudes reflect changes in blood flow in the facial vasculature. To determine pulse wave amplitude, it first summed hemoglobin signals from regions 10 through 17, and then normalized the signal by converting to a z-score. Next, we filtered this normalized signal through some of 60 different elliptic band pass filters (0.1 Hz width), ranging from 0 - 0.1 Hz to 5.9-6 Hz at 0.1 Hz increments – frequencies that are known to contain blood pressure information. We then filtered each signal through a low pass filter (0.02 Hz) to smooth the signal. The mean amplitude of each of these filtered signals was used as a feature.

Heart rate band pulse amplitudes were used as features to capture signals that closely corresponded to heart/pulse rate. To calculate these features, each region of interest from 10 through 17 was normalized by converting to a z-score and then summed. Signal outside of the 0.9 to 1.1 Hz band was filtered out, leaving only signal within the heart rate band. The technology then smoothed the signal by passing it through a low pass filter (0.02 Hz). The mean amplitude of the signal was then used as a feature.

Pulse rate across all regions was used as a feature. In each of the 17 regions of interest, the AI normalized signal by converting to a z-score. It then filtered each signal with a band pass filter to remove signal frequencies outside of the 0.7 to 2 Hz range. The data then subjected signals to a Hilbert-transform11, adjusted the signal for phase differences and then ‘unwrapped’ the signal so that it was no longer oscillatory in nature using MATLAB (The MathWorks, Inc.) functions. The AI then took the derivative of each signal and weighted it according to the standard deviation of the amplitude. Finally, signals from all regions of interest were combined and subjected to a low pass filter (0.02 Hz) to generate a single pulse rate feature.

Pulse rate variability calculated from an average of all regions of interest was used as a feature. The AI then calculated pulse rate by taking the derivative of the pulse rate from the average signal of all regions of interest. Lastly then BitDoctor AI will normalized the signal by converting to a z-score. Next, it filtered this signal through some of 20 elliptic band pass filters ranging from 0 - 0.01 to 0.38 - 0.4 Hz at 0.02 Hz increments (0.02 Hz width). Each of these signals was then filtered with a low pass filter (0.02 Hz) and then used to create a pulse rate variability feature.

Phase difference between regions of interest was used to calculate features indicative of pulse transit time (PTT). In blood pressure pulses, this is related to the speed at which the blood pressure wave propagates throughout the arterial vasculature. Evidence shows that pulse transit time across a constant distance is proportional to arterial stiffness, which in turn relates to systolic and diastolic blood pressure12. We normalized these signals by converting to a z-score. The AI then applied elliptic band pass filters – each corresponding to a frequency between 0 - 0.1 Hz and 5.9 - 6 Hz at 0.1 Hz increments and a width of 0.1 Hz. Then used filtered signals to calculate phase differences between selected regions of interest using signals filtered with each band pass filter to generate pulse transit time features (Supplemental Figure 1E).

Pulse shape features were used to describe the size of the pulse wave as well the shape of the pulse wave calculated in reference to commonly used pressure pulse landmarks10 (Supplemental Figure 1F). Features consisted of areas and distances within the pulse waveform, and were calculated on band pass filtered (0 – 6 Hz) signal. BitDoctor AI also calculated variations of these features that included detrended means, inverse means, standard deviations, maxima, minima and ratios of one feature with respect to another. Further, pulse energy features were calculated as the derivatives of these signal features to determine the maximal rate at which these features emerge as a function of time (Supplemental Figure 1G).

Meta features were used to help ‘normalize’ for different imaging conditions in the red, green and blue color channels such as light intensity, light gradient, light color, as well as features pertaining to ambient room temperature and subject physical characteristics (age, height, weight, gender, skin tone). In particular, skin tone was calculated by matching participant skin tone to the 6-point Fitzpatrick scale13, with 1 corresponding to light skin and 6 corresponding to dark skin. This was achieved by extracting the average forehead region colour in 10 different color spaces (sRGB, YCb, HSL, HSV, YUV, CIE Lab, linear RGB, YIQ, CIE XYZ and Gray) and then categorizing the color in each space into the closest colour category by minimizing vector distance. The AI then assigned each subject to the median category and used this category as a feature for modeling. This process was performed using an in-house program.

Rationale for evaluating models from multiple iterations BitDoctor.ai ran an identical training and testing process 200 times for each model. This is to assess the robustness of the models’ prediction accuracies and information gains. By doing so, it were able to measure the mean and variability of such accuracies and information gains of the models. The accuracy results we report for systolic, diastolic and pulse pressure models is the mean of these iterations with a corresponding measure of variability around this mean. Graphical representations of reference measurements versus predictions for systolic, diastolic and pulse pressure models show the mean prediction across all 200 iterations of that model. We also report the 95% confidence interval results in Tables 1.

Supplemental Figure

Suppl. Fig. 1 Schematic of BitDoctor AI Imaging blood flow feature calculation

Continuous AI Imaging signal from various regions of interest are processed using a variety of functions (filters, Hilbert, phase, unwrap, etc.) in MATLAB and then five types of features (A-G) are generated from the processed signal (126 blood flow features in total). Abbreviations: BPF, elliptic band pass filter; LPF, low pass filter (0.02 Hz); ROI, region of interest; SD, standard deviation; Hilbert, Hilbert transformation; Phase, processing of signal to remove phase differences, Unwrap, processing of signal so that it is no longer oscillatory in nature. Symbols: Plus sign, summation of signals; Dotted line, indicates a series of items; Division sign, division of signals.