# SensorKit **URL:** \ **Category:** Reference\ **Created:** [June 16, 2025, 9:26am UTC](https://forum.avicennaresearch.com/t/sensorkit/1290 "2025-06-16T09:26:30Z")\ **Posts on this page:** 1\ **Page:** 1
**Author:** ![mojtaba](https://forum.avicennaresearch.com/letter_avatar_proxy/v4/letter/m/e56c9b/32.png) [@mojtaba](https://forum.avicennaresearch.com/u/mojtaba)\ **Post date:** [June 16, 2025, 9:26am UTC](https://forum.avicennaresearch.com/t/sensorkit/1290/1 "2025-06-16T09:26:30Z")
[Apple SensorKit](https://developer.apple.com/documentation/sensorkit) provides access to rich sensor data from Apple devices, offering researchers detailed insights into raw data or metrics that the system processes from a sensor. This powerful framework enables the collection of data like device usage patterns, keyboard metrics, ambient light readings, pedometer data, and more. > [!note] > Avicenna currently supports SensorKit only on iPhones and paired Apple Watches. SensorKit is not supported on iPads. ## Adding SensorKit As a Data Source Access to SensorKit data is limited to research uses and requires a private entitlement, which Apple reviews separately for each study. To add SensorKit as a data source to your study, please [contact our support team](mailto:support@avicennareasearch.com). These are the main steps that should be done in the process: - A custom iOS application will be developed for your study (you can also have a custom Android application, but it’s not required). This is because our main application is designed to support all studies. For more details, you can check [our Branding documentation](https://forum.avicennaresearch.com/t/branding/1270/1). - After your custom iOS application is published in the App Store, it will have an App ID. - Then, you should fill out an entitlement request document with App ID, what data sources your study is going to collect, and why, etc. Also, you should attach the Institutional Review Board (IRB) approval to the document. For more detailed information about the SensorKit access process, visit [Apple’s Research & Care guide](https://www.researchandcare.org/resources/accessing-sensorkit-data/). ## Supported SensorKit Metrics in Avicenna Below is a comprehensive list of SensorKit metrics that Avicenna supports. We’ll first cover the common fields that appear across all SensorKit data types, followed by the specific fields for each individual metric. ### Common Fields Across All SensorKit Data Sources All metrics include [the common data fields](https://forum.avicennaresearch.com/t/data-sources/1266#p-2893-common-data-fields-4) and the following additional fields: #### Device Basic information about the device providing sensor data. Internally stored as part of each SensorKit record and includes: - **Name:** Device name (e.g., “John’s iPhone”). Internally stored as `name`. - **Model:** Device model identifier (e.g., “iPhone14,3” for iPhone 13 Pro Max). Internally stored as `model`. - **System Name:** Operating system name (e.g., “iOS”). Internally stored as `system_name`. - **System Version:** OS version number (e.g., “16.5.1”). Internally stored as `system_version`. - **Product Type (iOS 17.0+):** Device category. Internally stored as `product_type`. #### Source Device Type Type of device providing the measurement: - `UNKNOWN` (0) - `IPHONE` (1) - `WATCH` (2) - `IPAD` (3) - `OTHER` (4) ### SensorKit Ambient Light Provides information about environmental lighting conditions. Internally stored as `sensorkit_ambient_light` and includes: - **Illuminance Lux:** Describes the sample’s luminous flux. Internally stored as `illuminance_lux`. - **Chromaticity X:** Describes light brightness and tint x-value. Internally stored as `chromaticity_x_value`. - **Chromaticity Y:** Describes light brightness and tint y-value. Internally stored as `chromaticity_y_value`. - **Placement:** Sensor location on the device. Internally stored as `placement`. It can be one of the following: - `PLACEMENT_UNKNOWN` (0) - `FRONT_TOP` (1) - `FRONT_BOTTOM` (2) - `FRONT_RIGHT` (3) - `FRONT_LEFT` (4) - `FRONT_TOP_RIGHT` (5) - `FRONT_TOP_LEFT` (6) - `FRONT_BOTTOM_RIGHT` (7) - `FRONT_BOTTOM_LEFT` (8) ### SensorKit Pedometer [Download Sample Data](https://forum.avicennaresearch.com/uploads/short-url/chfstHno2pPGV9LVzFrf2QPotug.csv) Tracks pedestrian-related activity like steps taken and distance traveled. Internally stored as `sensorkit_pedometer` and includes: - **Start Date:** The start time for the pedometer data. Internally stored as `start_date`. - **End Date:** The end time for the pedometer data. Internally stored as `end_date`. - **Steps:** The number of steps taken by the user. Internally stored as `steps`. - **Distance:** The estimated distance traveled by the user in meters. Internally stored as `distance`. - **Average Active Pace:** The average pace of the user, measured in seconds per meter. Internally stored as `avg_active_pace`. - **Current Pace:** The current pace of the user, measured in seconds per meter. Internally stored as `current_pace`. - **Current Cadence:** The rate at which steps are taken, measured in steps per second. Internally stored as `current_cadence`. - **Floors Ascended:** The approximate number of floors ascended by walking. Internally stored as `floors_ascended`. - **Floors Descended:** The approximate number of floors descended by walking. Internally stored as `floors_descended`. > [!note] > SensorKit reports step counts as **cumulative values**. Each data point is a snapshot of the device’s running total steps since it was first activated. > Because these values represent a long-term total, the numbers will appear very large. To find the number of steps taken during a specific time, simply calculate the difference between the records at the start and end of that period. ### SensorKit Heart Rate Captures heart rate measurements in beats per minute (BPM) with associated confidence levels. Internally stored as `sensorkit_heart_rate` and includes: - **Heart Rate:** The heart rate value in units of beats per minute (BPM). Internally stored as `heart_rate`. - **Confidence:** The confidence level of the heart rate measurement. Internally stored as `confidence`. It can be one of the following: - `CONFIDENCE_UNKNOWN` (0) - `CONFIDENCE_LOW` (1) - `CONFIDENCE_MEDIUM` (2) - `CONFIDENCE_HIGH` (3) - `CONFIDENCE_HIGHEST` (4) ### SensorKit Accelerometer Measures device acceleration. Internally stored as `sensorkit_accelerometer` and includes: - **X-Axis:** X-axis acceleration in G’s (gravitational force). Internally stored as `x_axis`. - **Y-Axis:** Y-axis acceleration in G’s (gravitational force). Internally stored as `y_axis`. - **Z Axis:** Z-axis acceleration in G’s (gravitational force). Internally stored as `z_axis`. - **Identifier:** Unique measurement identifier. Internally stored as `identifier`. > [!note] > Data is downsampled to 32Hz (default is 100Hz). [Contact our support team](mailto:support@avicennareasearch.com) for custom sampling rate adjustments. ### SensorKit Rotation Rate Tracks device rotation. Internally stored as `sensorkit_rotation_rate` and includes: - **X-Axis:** Rotation rate around the x-axis. Internally stored as `x_axis`. - **Y-Axis:** Rotation rate around the y-axis. Internally stored as `y_axis`. - **Z-Axis:** Rotation rate around the z-axis. Internally stored as `z_axis`. > [!note] > Data is downsampled to 32Hz (default is 100Hz). [Contact our support team](mailto:support@avicennareasearch.com) for custom sampling rate adjustments. ### SensorKit Ambient Pressure Measures atmospheric pressure. Internally stored as `sensorkit_ambient_pressure` and includes: - **Identifier:** Unique measurement identifier. Internally stored as `identifier`. - **Pressure:** Atmospheric pressure measurement in hectopascals (hPa). Internally stored as `pressure_hpa`. - **Temperature:** Associated temperature measurement in Celsius. Internally stored as `temp_celsius`. ### SensorKit Message Usage Report Provides information about the use of the Messages app. Internally stored as `sensorkit_message_usage_report` and includes: - **Duration:** Report period in seconds. Internally stored as `duration_sec`. - **Total Incoming Messages:** Count of received messages. Internally stored as `total_incoming_messages`. - **Total Outgoing Messages:** Count of sent messages. Internally stored as `total_outgoing_messages`. - **Total Unique Contacts:** Number of distinct contacts. Internally stored as `total_unique_contacts`. ### SensorKit Phone Usage Report Reports the amount of time that the user is on phone calls. Internally stored as `sensorkit_phone_usage_report` and includes: - **Duration:** Report period in seconds. Internally stored as `duration_sec`. - **Total Incoming Calls:** Number of calls the user receives. Internally stored as `total_incoming_calls`. - **Total Outgoing Calls:** Number of calls the user makes. Internally stored as `total_outgoing_calls`. - **Total Phone Call Duration:** Total duration of phone calls in seconds. Internally stored as `total_phone_call_duration`. - **Total Unique Contacts:** Number of distinct contacts. Internally stored as `total_unique_contacts`. ### SensorKit Device Usage Report [Download Sample Data](https://forum.avicennaresearch.com/uploads/short-url/6eM9ctyuFzShPAeUn4E0X93Bwfl.csv) The frequency and relative duration that the user uses their device, particular Apple apps, or websites. Internally stored as `sensorkit_device_usage_report` and includes: - **Duration:** The duration that the report spans in seconds. Internally stored as `duration_sec`. - **Total Screen Wakes:** The total number of screen wakes for the device. Internally stored as `total_screen_wakes`. - **Total Unlocks:** The total number of unlocks for the device. Internally stored as `total_unlocks`. - **Total Unlock Duration:** The duration of time the device is in an unlocked state in seconds. Internally stored as `total_unlock_duration_sec`. - **Algorithm Version (iOS 16.4+):** The version of the algorithm that the system uses to generate the report. Internally stored as `algorithm_version`. - **App Usage By Category:** A list that describes the user’s app activity over a period of time grouped by category. Internally stored as `app_usage_by_category`. - **Category Key:** Categories of apps, notifications, or websites. Internally stored as `category_key`. It can be one of the following: - `CATEGORY_UNKNOWN` (0) - `BOOKS` (1) - `BUSINESS` (2) - `CATALOGS` (3) - `DEVELOPER_TOOLS` (4) - `EDUCATION` (5) - `ENTERTAINMENT` (6) - `FINANCE` (7) - `FOOD_AND_DRINK` (8) - `GAMES` (9) - `GRAPHICS_AND_DESIGN` (10) - `HEALTH_AND_FITNESS` (11) - `KIDS` (12) - `LIFESTYLE` (13) - `MEDICAL` (14) - `MISCELLANEOUS` (15) - `MUSIC` (16) - `NAVIGATION` (17) - `NEWS` (18) - `NEWS_STAND` (19) - `PHOTO_AND_VIDEO` (20) - `PRODUCTIVITY` (21) - `REFERENCE` (22) - `SHOPPING` (23) - `SOCIAL_NETWORKING` (24) - `SPORTS` (25) - `STICKERS` (26) - `TRAVEL` (27) - `UTILITIES` (28) - `WEATHER` (29) - **App Usage:** Per-app usage details. Internally stored as `app_usage`. - **Bundle Identifier:** The bundle identifier of the app in use (if available). Internally stored as `bundle_identifier`. - **Report App Identifier (iOS 15.0+):** A pseudonym for a real application identifier. Internally stored as `report_app_identifier`. - **Supplemental Categories (iOS 16.4+):** Categories that provide more information about an app. Internally stored as `supplemental_categories`. - **Identifier:** List of unique identifiers for the supplemental category. Internally stored as `identifier`. - **Usage Time:** The amount of time the user uses the app in seconds. Internally stored as `usage_time_sec`. - **Relative Start Time (iOS 16.4+):** The time the user starts the app relative to the start time of the first app in a report interval in seconds. Internally stored as `relative_start_time_sec`. - **Text Input Sessions (iOS 15.0+):** List of text input session types that occur during application usage. Internally stored as `text_input_sessions`. - **Session Identifier:** A unique identifier for the keyboard session. Internally stored as `session_identifier`. - **Duration:** The length of time, in seconds, that the session spans. Internally stored as `duration_sec`. - **Session Type:** Methods to input text during a session. Internally stored as `session_type`. It can be one of the following: - `TYPE_UNKNOWN` (0) - `KEYBOARD` (1) - `THIRD_PARTY_KEYBOARD` (2) - `PENCIL` (3) - `DICTATION` (4) - **Notification Usage By Category:** List of the frequency of notifications per category. Internally stored as `notification_usage_by_category`. - **Category Key:** Application category. Internally stored as `category_key`. It can be one of the values same as the `Category Key` above. - **Notification Usage:** Per-notification details. Internally stored as `notification_usage`. - **Bundle Identifier:** The bundle identifier of the app that corresponds to the notification. Internally stored as `bundle_identifier`. - **Event:** The way that the user interacts with the notification. Internally stored as `event`. It can be one of the following: - `EVENT_UNKNOWN` (0) - `RECEIVED` (1) - `DEFAULT_ACTION` (2) - `SUPPLEMENTARY_ACTION` (3) - `CLEAR` (4) - `CLEAR_ALL` (5) - `REMOVED` (6) - `HIDE` (7) - `LONG_LOOK` (8) - `SILENCE` (9) - `APP_LAUNCH` (10) - `EXPIRED` (11) - `BANNER_PULL_DOWN` (12) - `TAP_COALESCE` (13) - `DEDUPED` (14) - `DEVICE_ACTIVATED` (15) - `DEVICE_UNLOCKED` (16) - **Web Usage By Category:** The amount of time the user accesses domains per category. Internally stored as `web_usage_by_category`. - **Category Key:** Website category. Internally stored as `category_key`. It can be one of the values same as the `Category Key` above. - **Web Usage:** Per-category web usage. Internally stored as `web_usage`. - **Total Usage Time:** The amount of web usage time that the report spans. Internally stored as `total_usage_time_sec`. ### SensorKit Visits Provides information about frequently visited locations. Internally stored as `sensorkit_visits` and includes: - **Identifier:** A value that maps to a unique geographic location. Internally stored as `identifier`. - **Arrival Start Date:** The start date of time within which the user arrives at a location of interest. Internally stored as `arrival_start_date`. - **Arrival End Date:** The end date of time within which the user arrives at a location of interest. Internally stored as `arrival_end_date`. - **Departure Start Date:** The start date of time within which the user departs from a location of interest. Internally stored as `departure_start_date`. - **Departure End Date:** The end date of time within which the user departs from a location of interest. Internally stored as `departure_end_date`. - **Distance from Home:** The location’s distance from the home-category location in meters. Internally stored as `distance_from_home`. - **Location Category:** Type of location. Internally stored as `location_category`. It can be one of the following: - `LOCATION_UNKNOWN` (0) - `HOME` (1) - `WORK` (2) - `SCHOOL` (3) - `GYM` (4) ### SensorKit On-Wrist State Tracks Apple Watch wear patterns. Internally stored as `sensorkit_on_wrist_state` and includes: - **Crown Orientation:** A value that indicates the direction the Digital Crown (of Apple Watch) faces with respect to the user. Internally stored as `crown_orientation`. It can be one of the following: - `ORIENTATION_LEFT` (0) - `ORIENTATION_RIGHT` (1) - `ORIENTATION_UNKNOWN` (2) - **On Wrist:** Whether the watch is currently being worn. Internally stored as `on_wrist`. - **Wrist Location:** Which wrist the watch is worn on. Internally stored as `wrist_location`. It can be one of the following: - `LOCATION_LEFT` (0) - `LOCATION_RIGHT` (1) - `LOCATION_UNKNOWN` (2) - **On Wrist Date (iOS 17.0+):** The Timestamp when the watch was put on the wrist. Internally stored as `on_wrist_date`. - **Off Wrist Date (iOS 17.0+):** Timestamp when the watch was removed from the wrist. Internally stored as `off_wrist_date`. ### SensorKit Wrist Temperature Provides wrist temperature while the user sleeps. Internally stored as `sensorkit_wrist_temperature` and includes: - **Temperature:** The temperature sensor value in Celsius. Internally stored as `temperature_celsius`. - **Error Estimate:** An estimate of the amount of error in the temperature measurement in Celsius. Internally stored as `error_estimate`. The error could be in either positive or negative direction. - **Conditions:** The conditions of the measurement that impact its accuracy. Internally stored as `conditions`. It can be one of the following: - `CONDITION_UNKNOWN` (0) - `CONDITION_OFF_WRIST` (1) - `CONDITION_ON_CHARGER` (2) - `CONDITION_IN_MOTION` (3) ### SensorKit Siri Speech Metrics > iOS 17.0+ These metrics provide details about the user’s voice, such as tenor, pitch, cadence, and speech timing, which includes words per minute and the average duration between words. This sensor doesn’t provide raw audio data. For user privacy, SensorKit removes any transcript strings from the result. Internally stored as `sensorkit_speech_metrics` and includes: - **Session Identifier:** An identifier for the audio session. Internally stored as `session_identifier`. - **Time Since Audio Start (iOS 17.2+):** The number of seconds since the start of the audio stream in seconds. Internally stored as `time_since_audio_start`. - **Audio Level:** The audio level of the speech if available. Internally stored as `audio_level`. - **Loudness:** The measure of the audio level in decibels. Internally stored as `loudness`. - **Start Time Range:** The start time of the audio stream that the level applies to in seconds. Internally stored as `start_time_range_sec`. - **End Time Range:** The end time of the audio stream that the level applies to in seconds. Internally stored as `end_time_range_sec`. - **Speech Recognition Results:** The partial or final results of the speech recognition request (if available). Internally stored as `speech_recognition_results`. - **Best Transcription:** The transcription with the highest confidence level. Internally stored as `best_transcription`. - **Formatted String:** The entire transcription of utterances, formatted into a single, user-displayable string. Internally stored as `formatted_string`. - **Segments:** An array of transcription segments that represent the parts of the transcription, as identified by the speech recognizer. Internally stored as `segments`. - **Substring:** The string representation of the utterance in the transcription segment. Internally stored as `substring`. - **Alternative Substrings:** An array of alternate interpretations of the utterance in the transcription segment. Internally stored as `alternative_substrings`. - **Substring Range Lower Bound:** The lower bound of the substring range in the transcription segment. Internally stored as `substring_range_lower_bound`. - **Substring Range Upper Bound:** The upper bound of the substring range in the transcription segment. Internally stored as `substring_range_upper_bound`. - **Confidence:** The level of confidence the speech recognizer has in its recognition of the speech transcribed for the segment. Internally stored as `confidence`. - **Timestamp:** The start time of the segment in the processed audio stream. Internally stored as `timestamp`. - **Duration:** The number of seconds it took for the user to speak the utterance represented by the segment. Internally stored as `duration`. - **Transcriptions:** An array of potential transcriptions, sorted in descending order of confidence. Internally stored as `transcriptions`. This contains the same fields as the `Best Transcription` above. - **Is Final:** A Boolean value that indicates whether speech recognition is complete and whether the transcriptions are final. Internally stored as `is_final`. - **Metadata:** Contains the metadata results for a speech recognition request (if available). Internally stored as `metadata`. - **Average Pause Duration:** The average pause duration between words, measured in seconds. Internally stored as `average_pause_duration`. - **Speaking Rate:** The number of words spoken per minute. Internally stored as `speaking_rate`. - **Speech Duration:** The duration in seconds of speech in the audio. Internally stored as `speech_duration`. - **Speech Start Timestamp:** The start timestamp of speech in the audio. Internally stored as `speech_start_timestamp`. - **Voice Analytics:** An analysis of the transcription segment’s vocal properties (if available). Internally stored as `voice_analytics`. - **Voicing:** The likelihood of a voice in each frame of a transcription segment. Internally stored as `voicing`. - **Frame Duration:** The duration of the audio frame in seconds. Internally stored as `frame_duration`. - **Acoustic Feature Value Per Frame:** An array of feature values, one value per audio frame, corresponding to a transcript segment of recorded audio. Internally stored as `acoustic_feature_value_per_frame`. - **Pitch:** The highness or lowness of the tone (fundamental frequency) in each frame of a transcription segment, expressed as a logarithm. Internally stored as `pitch`. It contains the same fields as the `Voicing` above. - **Jitter:** The variation in pitch in each frame of a transcription segment, expressed as a percentage of the frame’s fundamental frequency. Internally stored as `jitter`. It contains the same fields as the `Voicing` above. - **Shimmer:** The variation in vocal volume stability (amplitude) in each frame of a transcription segment, expressed in decibels. Internally stored as `shimmer`. It contains the same fields as the `Voicing` above. - **Sound Classification:** The highest-ranking classifications in the time range (if available). Internally stored as `speech_classification`. - **Start Time Range:** The start time span that corresponds to the result’s classifications. Internally stored as `start_time_range`. - **End Time Range:** The end time span that corresponds to the result’s classifications. Internally stored as `end_time_range`. - **Classifications:** A sorted array of the request’s top classification candidates. Internally stored as `classifications`. - **Identifier:** A prediction label that’s one of the classifications a sound classifier’s underlying model defines. Internally stored as `identifier`. - **Confidence:** The confidence value the model has in its prediction. Internally stored as `confidence`. - **Speech Expression:** The metrics and voice analytics for the range of speech (if available). Internally stored as `speech_expression`. - **Start Time Range:** The start time in the audio stream that the metrics and analytics apply to. Internally stored as `start_time_range`. - **End Time Range:** The end time in the audio stream that the metrics and analytics apply to. Internally stored as `end_time_range`. - **Version:** The version of the algorithm that the system uses to generate the metrics and analytics. Internally stored as `version`. - **Confidence:** The level of confidence of the speaker. Internally stored as `confidence`. - **Mood:** An indication of how slurry, tired, or exhausted the speaker sounds compared to normal speech. Internally stored as `mood`. - **Valence:** The degree of positive or negative emotion or sentiment of the speaker. Internally stored as `valence`. - **Activation:** The level of energy or activation of the speaker. Internally stored as `activation`. - **Dominance:** The degree of how strong or meek the speaker sounds. Internally stored as `dominance`. ### SensorKit Telephony Speech Metrics > iOS 17.0+ Provides data describing speech during phone calls. See [SensorKit Siri Speech Metrics](#p-2917-sensorkit-siri-speech-metrics-18) for detailed field descriptions. ### SensorKit Keyboard Metrics Provides information about keyboard usage. Internally stored as `sensorkit_keyboard_metrics` and includes: - **Duration:** Duration of keyboard usage in seconds. Internally stored as `duration_sec`. - **Keyboard Identifier:** The identifier of the keyboard in the keyboard list. Internally stored as `keyboard_identifier`. - **Version:** The version of keyboard metrics. Internally stored as `version`. - **Width:** The width, in millimeters, of the keyboard in the report. Internally stored as `width_mm`. - **Height:** The height, in millimeters, of the keyboard in the report. Internally stored as `height_mm`. - **Input Modes (iOS 15.0+):** List of The active keyboard languages in the session. Internally stored as `input_modes`. - **Session Identifiers (iOS 16.4+):** List of identifiers for the keyboard sessions that report metrics to the sample. Internally stored as `session_identifiers`. - **Total Words:** The total number of typed words for the keyboard. Internally stored as `total_words`. - **Total Altered Words:** The total number of words that were modified. Internally stored as `total_altered_words`. - **Total Taps:** The total number of key taps. Internally stored as `total_taps`. - **Total Drags:** The total number of drag gestures. Internally stored as `total_drags`. - **Total Deletes:** The total number of delete operations. Internally stored as `total_deletes`. - **Total Emojis:** The total number of emojis used. Internally stored as `total_emojis`. - **Total Paths:** The total number of swipe-typing paths. Internally stored as `total_paths`. - **Total Path Time:** The total time to complete paths for the keyboard in seconds. Internally stored as `total_path_time_sec`. - **Total Path Length:** The total length of completed paths for the keyboard in centimeters. Internally stored as `total_path_length_cm`. - **Total Auto Corrections:** The total number of automatic corrections. Internally stored as `total_auto_corrections`. - **Total Space Corrections:** The total number of corrections made using the space key. Internally stored as `total_space_corrections`. - **Total Retro Corrections:** The total number of retroactive corrections. Internally stored as `total_retro_corrections`. - **Total Transposition Corrections:** The total number of corrections of transposed characters. Internally stored as `total_transposition_corrections`. - **Total Insert Key Corrections:** The total number of corrections using insert key. Internally stored as `total_insert_key_corrections`. - **Total Skip Touch Corrections:** The total number of skipped touch corrections. Internally stored as `total_skip_touch_corrections`. - **Total Near Key Corrections:** The total number of corrections of nearby key presses. Internally stored as `total_near_key_corrections`. - **Total Substitution Corrections:** The total number of character substitution corrections. Internally stored as `total_substitution_corrections`. - **Total Hit Test Corrections:** The total number of hit test corrections for the keyboard. Internally stored as `total_hit_test_corrections`. - **Total Typing Duration:** Overall time spent typing in seconds. Internally stored as `total_typing_duration_sec`. - **Total Path Pauses (iOS 15.0+):** The total number of pauses while drawing a path for a word. Internally stored as `total_path_pauses`. - **Total Pauses (iOS 15.0+):** The total number of pauses during the session. Internally stored as `total_pauses`. - **Total Typing Episodes (iOS 15.0+):** The total number of continuous typing episodes during the session. Internally stored as `total_typing_episodes`. - **Touch Down Up:** The duration between touch-down to touch-up for any key in milliseconds. Internally stored as `touch_down_up_ms`. - **Distribution Sample Values:** An array of sample values from the probability distribution. Internally stored as `distribution_sample_values` - **Touch Up Down (iOS 16.4+):** The duration between key release and next press in milliseconds. Internally stored as `touch_up_down_ms`. It contains the same fields as the `Touch Down Up` above. - **Space Touch Down Up:** The time between space key press and release in milliseconds. Internally stored as `space_touch_down_up_ms`. It contains the same fields as the `Touch Down Up` above. - **Space to Space Key:** The duration between consecutive space key presses in milliseconds. Internally stored as `space_to_space_key_ms`. It contains the same fields as the `Touch Down Up` above. - **Space to Delete Key:** The duration between the touch-up of the Space bar and the touch-down of a sequential Delete key in milliseconds. Internally stored as `space_to_delete_key_ms`. It contains the same fields as the `Touch Down Up` above. - **Space to Char Key:** The duration between the touch-up of the Space bar and the touch-down of a character key in milliseconds. Internally stored as `space_to_char_key_ms`. It contains the same fields as the `Touch Down Up` above. - **Space to Shift Key:** The time from space to shift key in milliseconds. Internally stored as `space_to_shift_key_ms`. It contains the same fields as the `Touch Down Up` above. - **Space to Plane Change Key:** The time from space to keyboard plane changes in milliseconds. Internally stored as `space_to_plane_change_key_ms`. It contains the same fields as the `Touch Down Up` above. - **Space to Prediction Key:** The time from space to prediction selection in milliseconds. Internally stored as `space_to_prediction_key_ms`. It contains the same fields as the `Touch Down Up` above. - **Space to Path:** The time from space to swipe-typing in milliseconds. Internally stored as `space_to_path_ms`. It contains the same fields as the `Touch Down Up` above. - **Space Up Error Distance:** The upward touch accuracy error for the space key in millimeters. Internally stored as `space_up_error_distance_mm`. It contains the same fields as the `Touch Down Up` above. - **Space Down Error Distance:** The downward touch accuracy error for the space key in millimeters. Internally stored as `space_down_error_distance_mm`. It contains the same fields as the `Touch Down Up` above. - **Delete Touch Down Up:** The time between the delete key press and release in milliseconds. Internally stored as `delete_touch_down_up_ms`. It contains the same fields as the `Touch Down Up` above. - **Delete to Space Key:** The time from delete to space key in milliseconds. Internally stored as `delete_to_space_key_ms`. It contains the same fields as the `Touch Down Up` above. - **Delete to Delete:** The time between single delete key presses in milliseconds. Internally stored as `delete_to_delete_ms`. It contains the same fields as the `Touch Down Up` above. - **Delete to Deletes:** The time between multiple delete presses in milliseconds. Internally stored as `delete_to_deletes_ms`. It contains the same fields as the `Touch Down Up` above. - **Delete to Char Key:** The time from delete to character keys in milliseconds. Internally stored as `delete_to_char_key_ms`. It contains the same fields as the `Touch Down Up` above. - **Delete to Shift Key:** The time from delete to shift key in milliseconds. Internally stored as `delete_to_shift_key_ms`. It contains the same fields as the `Touch Down Up` above. - **Delete to Plane Change Key:** Time from delete to keyboard plane changes in milliseconds. Internally stored as `delete_to_plane_change_key_ms`. It contains the same fields as the `Touch Down Up` above. - **Delete to Path:** Time from delete to swipe-typing in milliseconds. Internally stored as `delete_to_path_ms`. It contains the same fields as the `Touch Down Up` above. - **Delete Up Error Distance:** Upward touch accuracy error for delete key in millimeters. Internally stored as `delete_up_error_distance_mm`. It contains the same fields as the `Touch Down Up` above. - **Delete Down Error Distance:** Downward touch accuracy error for delete key in millimeters. Internally stored as `delete_down_error_distance_mm`. It contains the same fields as the `Touch Down Up` above. - **Char Key to Space Key:** Time from character to space key in milliseconds. Internally stored as `char_key_to_space_key_ms`. It contains the same fields as the `Touch Down Up` above. - **Char Key to Delete:** Time from character to delete key in milliseconds. Internally stored as `char_key_to_delete_ms`. It contains the same fields as the `Touch Down Up` above. - **Char Key to Plane Change Key:** Time from character to keyboard plane changes in milliseconds. Internally stored as `char_key_to_plane_change_key_ms`. It contains the same fields as the `Touch Down Up` above. - **Char Key to Prediction:** Time from character to prediction selection in milliseconds. Internally stored as `char_key_to_prediction_ms`. It contains the same fields as the `Touch Down Up` above. - **Char Key to Any Tap Key:** Time from character to any key tap in milliseconds. Internally stored as `char_key_to_any_tap_ms`. It contains the same fields as the `Touch Down Up` above. - **Short Word Char Key Touch Down Up:** Press-release time for short word characters in milliseconds. Internally stored as `short_word_char_key_touch_down_up_ms`. It contains the same fields as the `Touch Down Up` above. - **Short Word Char Key to Char Key:** Time between characters in short words in milliseconds. Internally stored as `short_word_char_key_to_char_key_ms`. It contains the same fields as the `Touch Down Up` above. - **Short Word Char Key Up Error Distance:** Upward touch accuracy error in short words in millimeters. Internally stored as `short_word_char_key_up_error_distance_mm`. It contains the same fields as the `Touch Down Up` above. - **Short Word Char Key Down Error Distance:** Downward touch accuracy error in short words in millimeters. Internally stored as `short_word_char_key_down_error_distance_mm`. It contains the same fields as the `Touch Down Up` above. - **Any Tap to Char Key:** Time from any tap to character key in milliseconds. Internally stored as `any_tap_to_char_key_ms`. It contains the same fields as the `Touch Down Up` above. - **Any Tap to Plane Change Key:** Time from any tap to plane change in milliseconds. Internally stored as `any_tap_to_plane_change_key_ms`. It contains the same fields as the `Touch Down Up` above. - **Plane Change Key to Char Key:** Time from plane change to character key in milliseconds. Internally stored as `plane_change_key_to_char_key_ms`. It contains the same fields as the `Touch Down Up` above. - **Plane Change to Any Tap:** Time from plane change to any tap in milliseconds. Internally stored as `plane_change_key_to_any_tap_ms`. It contains the same fields as the `Touch Down Up` above. - **Path to Space:** The duration between the touch-up of a path and the touch-down of a sequential Space bar in Milliseconds. Internally stored as `path_to_space_ms`. It contains the same fields as the `Touch Down Up` above. - **Path to Delete:** The duration between the touch-up of a path and the touch-down of a sequential Delete key in Milliseconds. Internally stored as `path_to_delete_ms`. It contains the same fields as the `Touch Down Up` above. - **Path to Path:** Time between consecutive paths in milliseconds. Internally stored as `path_to_path_ms`. It contains the same fields as the `Touch Down Up` above. - **Up Error Distance:** General upward touch accuracy error in millimeters. Internally stored as `path_up_error_distance_mm`. It contains the same fields as the `Touch Down Up` above. - **Down Error Distance:** General downward touch accuracy error in millimeters. Internally stored as `path_down_error_distance_mm`. It contains the same fields as the `Touch Down Up` above. - **Long Word Touch Down Up:** Distribution of press-release times for long words in milliseconds. Internally stored as `long_word_touch_down_up_ms`. It contains the same fields as the `Touch Down Up` above. - **Long Word Touch Down Down:** Distribution of times between presses in long words in milliseconds. Internally stored as `long_word_touch_down_down_ms`. It contains the same fields as the `Touch Down Up` above. - **Long Word Up Error Distance:** The distance from the touch-up to the center of the intended key of the characters of a long word in millimeters. Internally stored as `long_word_up_error_distance_mm`. It contains the same fields as the `Touch Down Up` above. - **Long Word Down Error Distance:** The distance from the touch-down to the center of the intended key of the characters of a long word in millimeters. Internally stored as `long_word_down_error_distance_mm`. It contains the same fields as the `Touch Down Up` above. - **Touch Down Down:** Time between consecutive key presses in milliseconds. Internally stored as `long_word_touch_down_down_ms`. It contains the same fields as the `Touch Down Up` above. - **Path Typing Speed (iOS 15.0+):** The QuickType words per minute in the session. Internally stored as `path_typing_speed_words_per_min`. - **Typing Speed Char (iOS 15.0+):** The user’s typing rate in characters per second. Internally stored as `typing_speed_char_per_sec`. - **Path Error Distance Ratio:** List of sample values of the ratio of error distance between the intended and actual path. Internally stored as `path_error_distance_ratio`. - **Word Count Sentiments:** Analyzes the emotional content of typed words and provides counts for different sentiment categories. Internally stored as `word_count`. Possible values are: - `ABSOLUTIST` (0): Words expressing absolute or black-and-white thinking (e.g., “always”, “never”) - `DOWN` (1): Words indicating feeling down or depressed - `DEATH` (2): References to death, dying, or mortality - `ANXIETY` (3): Words expressing worry, fear, or anxiety - `ANGER` (4): Words conveying anger, frustration, or aggression - `HEALTH` (5): References to health, illness, or medical conditions - `POSITIVE` (6): Words expressing optimism, joy, or positive emotions - `SAD` (7): Words indicating sadness or grief - `LOW_ENERGY` (8): Words suggesting fatigue, exhaustion, or low motivation - `CONFUSED` (9): Words expressing confusion, uncertainty, or disorientation - `CATEGORY_UNKNOWN` (10): Words that don’t fit into other sentiment categories - **Emoji Count Sentiments:** Tracks the frequency of emoji usage categorized by emotional sentiment, providing insight into non-verbal emotional expression in digital communication. Uses the same sentiment categories as `Word Count Sentiments`. ### SensorKit Odometer > iOS 17.0+ Provides data about distance-related activities, specifically information about the distance traveled. Internally stored as `sensorkit_odometer` and includes: - **Delta Distance Meters:** The change in distance that the user travels since the last location, measured in meters. Internally stored as `delta_distance_meters`. - **Delta Distance Accuracy:** The accuracy of the change in distance value. Internally stored as `delta_distance_accuracy`. - **Speed:** The instantaneous velocity of the device, measured in meters per second. Internally stored as `speed`. - **Speed Accuracy:** The accuracy of the speed measurement. Internally stored as `speed_accuracy`. - **Slope:** The slope at the location toward the direction of travel, measured in degrees. Internally stored as `slope`. - **Max Absolute Slope:** The maximum absolute slope at the location toward all directions, measured in degrees. Internally stored as `maxabsslope`. - **GPS Date:** The time of the GPS measurement associated with the location. Internally stored as `gps_date`. - **Delta Altitude Meters:** The change in altitude above mean sea level associated with the location, measured in meters. Internally stored as `delta_altitude_meters`. - **Vertical Accuracy:** The validity of the altitude values and their estimated uncertainty, measured in meters. Internally stored as `vertical_accuracy`. - **Origin Device Type:** The type of device that the odometer data was collected from. Internally stored as `origin_device_type`. It can be one of the following: - `UNKNOWN` (0) - `LOCAL` (1): The origin of the odometer sample comes from the same device that requests the sample. - `REMOTE` (2): The origin of the odometer sample comes from a device that’s paired with the local device. - **Start Date:** The time that the device starts recording the odometer data. Internally stored as `start_date`. - **End Date:** The time that the device stops recording the odometer data. Internally stored as `end_date`. ### SensorKit Media Events > iOS 16.4+ Provides data about user interaction with a media object, such as an image or a video. Internally stored as `sensorkit_media_events` and includes: - **Event Type:** The type of user interaction with the media object. Internally stored as `event_type`. Can be one of the following: - `SCREEN_ON` (0): Media content appeared on screen. - `SCREEN_OFF` (1): Media content disappeared from screen. - **Media Identifier:** Unique identifier for the media content. Internally stored as `media_identifier`. ### SensorKit Face Metrics > iOS 17.0+ Provides metrics about the user’s face. Internally stored as `sensorkit_face_metrics` and includes: - **Version:** The version of the algorithm that the system uses to generate the face metrics and analytics. Internally stored as `version`. - **Session Identifier:** An identifier for the camera session. Internally stored as `session_identifier`. - **Context:** The context in which the face metrics were collected. Internally stored as `context`. Possible values are: - `UNKNOWN` (0) - `DEVICE_UNLOCK` (1): Collected during device unlock - `MESSAGING_APP_USAGE` (2): Collected during messaging app usage - **Whole Face Expressions:** Array of facial expression measurements for the entire face. Internally stored as `whole_face_expressions`. Each expression contains: - **Identifier:** The identifier for the facial expression. Internally stored as `identifier`. - **Value:** The current position of the expression. Internally stored as `value`. - **Partial Face Expressions:** Array of facial expression measurements for partial face analysis. Internally stored as `partial_face_expressions`. Each expression contains the same fields as `Whole Face Expressions`. - **Face Anchor:** The anchor for the face that the sensor detects in front of the camera. Internally stored as `face_anchor`. It contains the following fields: - **Transform:** A 4x4 transformation matrix representing the overall face pose and position. Internally stored as `transform`. - **Left Eye Transform:** A 4x4 transformation matrix for the left eye position and orientation. Internally stored as `left_eye_transform`. - **Right Eye Transform:** A 4x4 transformation matrix for the right eye position and orientation. Internally stored as `right_eye_transform`. - **Look At Point:** A position in face coordinate space estimating the direction of the face’s gaze. Internally stored as `look_at_point`. - **Blend Shapes:** A dictionary of named coefficients representing the detected facial expression in terms of the movement of specific facial features. Internally stored as `blend_shapes`. - **Geometry:** A coarse triangle mesh representing the topology of the detected face. Internally stored as `geometry` and contains: - **Vertices:** An array of vertex positions for each point in the face mesh. Internally stored as `vertices`. - **Texture Coordinates:** An array of texture coordinate values for each point in the face mesh. Internally stored as `texture_coordinates`. - **Triangle Indices:** An array of indices describing the triangle mesh formed by the face geometry’s vertex data. Internally stored as `triangle_indices`. ### SensorKit PPG Provides photoplethysmography (optical pulse waveform) measurements. Internally stored as `sensorkit_ppg` and includes: - **Measurement ID:** Unique identifier for each PPG measurement window. Internally stored as `measurement_id`. - **Session Identifier:** Identifier for the PPG recording session. Internally stored as `session_identifier`. - **Start Date:** Timestamp when PPG collection starts. Internally stored as `start_date`. - **End Date:** Timestamp when PPG collection ends. Internally stored as `end_date`. - **Sampling Frequency:** PPG sampling rate, in Hz. Internally stored as `sampling_frequency_hz`. - **Channel:** Optical channel/source (if available). Internally stored as `channel`. - **Wavelength:** Light wavelength used for the channel, in nanometers (nm). Internally stored as `wavelength_nm`. - **Raw Signal:** Raw PPG waveform sample values. Internally stored as `raw_signal`. - **Filtered Signal:** Preprocessed/filtered waveform values (if available). Internally stored as `filtered_signal`. - **Signal Quality Index:** Quality score for waveform reliability. Internally stored as `signal_quality_index`. - **Motion Artifact Flag:** Indicates potential motion contamination. Internally stored as `motion_artifact_flag`. - **Perfusion Index:** Relative pulse strength/perfusion measure (if available). Internally stored as `perfusion_index`. - **Pulse Rate:** Pulse estimate derived from PPG, in beats per minute (BPM). Internally stored as `pulse_rate_bpm`. - **Inter-Beat Intervals:** Beat-to-beat intervals, in milliseconds (ms). Internally stored as `inter_beat_intervals_ms`. - **Confidence:** Confidence level of derived measurements. Internally stored as `confidence`. - **Wear State:** Indicates whether the watch was on-wrist during measurement. Internally stored as `wear_state`. ### SensorKit ECG Provides electrocardiogram-related measurements. Internally stored as `sensorkit_ecg` and includes: - **Measurement ID:** Unique identifier for each ECG measurement. Internally stored as `measurement_id`. - **Session Identifier:** Identifier for the ECG recording session. Internally stored as `session_identifier`. - **Start Date:** Timestamp when ECG recording starts. Internally stored as `start_date`. - **End Date:** Timestamp when ECG recording ends. Internally stored as `end_date`. - **Sampling Frequency:** ECG sampling rate, in Hz. Internally stored as `sampling_frequency_hz`. - **Lead Number:** Lead configuration used for the ECG recording. Internally stored as `lead_number`. - **Classification:** Algorithm-derived rhythm classification. Internally stored as `classification`. Possible values are: - `CLASSIFICATION_UNKNOWN` (0) - `SINUS_RHYTHM` (1) - `ATRIAL_FIBRILLATION` (2) - `HIGH_HEART_RATE` (3) - `LOW_HEART_RATE` (4) - `INCONCLUSIVE` (5) - **Average Heart Rate:** Average heart rate during ECG capture, in beats per minute (BPM). Internally stored as `avg_heart_rate_bpm`. - **Symptoms Present:** Indicates if symptoms were reported during recording. Internally stored as `symptoms_present`. - **Voltage Samples:** ECG waveform samples, in millivolts (mV). Internally stored as `voltage_mv`. - **Sample Timestamps:** Relative or absolute timestamps for waveform samples. Internally stored as `sample_timestamps`. - **Quality Flags:** Quality indicators for motion/noise/contact issues. Internally stored as `quality_flags`. - **Device Placement:** Body/device placement context if provided by framework. Internally stored as `device_placement`. ## Data Collection Behavior Avicenna’s SensorKit implementation operates within the app’s primary data collection cycle (1-minute active collection every 5 minutes) to efficiently gather sensor data from iOS devices. During each active collection window, the framework fetches data using a sophisticated dual-cycle approach: most sensors operate on 8-hour collection windows, while high-frequency sensors (accelerometer and rotation) use 1-hour windows repeated 8 times before cycling to the next sensor. This design optimizes data collection while managing the system resources. The framework intelligently tracks the last successful fetch for each sensor-device combination to ensure continuous data coverage, implementing automatic downsampling to 32Hz for high-frequency sensors to maintain data quality while minimizing storage requirements. When the app enters its 1-minute collection window, SensorKit initiates fetch requests for any pending data since the last successful fetch, ensuring seamless integration with Avicenna’s broader data collection strategy while adhering to iOS’s privacy and performance guidelines. > [!note] > SensorKit can only access collected data after a delay of 24 hours due to Apple’s privacy policies. To account for known SensorKit bugs, we apply a **30-hour threshold** , which includes the 24-hour wait time plus an additional 6-hour buffer. As a result, any data collected now will be available for export **approximately 30 hours later**. ## Monitoring and Exporting SensorKit Data You can export and download the collected SensorKit data using [the Data Export page](https://forum.avicennaresearch.com/t/raw-data/1241#p-2868-exporting-raw-data-1).