All about macOS event observation

Takayama Fumihiko

https://pqrs.org/

Karabiner-Elements author

​

Jul 7, 2019

Overview

• macOS provides several APIs for event observing.
• [NSView keyDown]
• [NSApplication sendEvent]
• CGEventTapCreate, [NSEvent addGlobalMonitorForEvents]
• IOHIDQueueRegisterValueAvailableCallback
• etc.
• macOS Catalina (10.15) will require a user approval of input monitoring for security. (Congratulation!)

​

Introduction

Input event observing essentials

• Security
• The observer will become keylogger. The security is most important.�
• Completeness
• Coverage of input events.�(e.g., cannot handle "Internet" keys.)�
• Coverage of situation.�(e.g., cannot handle during Secure Keyboard Entry.)�
• Stability
• Of course, it’s good the observer is stable.

​

Typical Usage

• Hotkey (e.g., open launcher by command-space)
• Text Expander (e.g., replacing `\now` with `2019-01-01 ...` in any editors.)
• Key remapper
• Keylogger (malicious app!)

​

macOS event observation methods

• Observe events for own process
• [NSView keyDown]
• [NSApplication sendEvent]�[NSEvent addLocalMonitorForEvents]
• Observe events for other processes
• CGEventTapCreate, [NSEvent addGlobalMonitorForEvents]
• IOKit IOHIDDevice
• IOHIDQueueRegisterValueAvailableCallback
• IOHIDDeviceRegisterInputReportCallback
• Input source
• Custom device driver
• IOHIKeyboard function pointer replacement
• Method swizzling + [NSApplication sendEvent] (e.g., SIMBL)

Detail and demonstration

[NSView keyDown]

[NSApplication sendEvent]

[NSView keyDown], [NSApplication sendEvent]

• Pros
• Can receive any events for own process
• Can modify the received events.
• root permission is not required.
• Cons
• Cannot receive events for other processes.
• Note
• User approval is not required.

Demonstration: [NSView keyDown]

nsview-example

( https://github.com/tekezo/osx-event-observer-examples )

​

Code: [NSView keyDown]

ExampleView.swift:

​

class ExampleView: NSView {

override var acceptsFirstResponder: Bool { return true }

​

override func keyDown(with event: NSEvent) {

}

​

override func keyUp(with event: NSEvent) {

}

​

override func flagsChanged(with event: NSEvent) {

}

}

Demonstration: [NSApplication sendEvent]

nsapplication-example

( https://github.com/tekezo/osx-event-observer-examples )

​

Code: [NSApplication sendEvent]

ExampleApplication.swift:

​

class ExampleApplication: NSApplication {

override func sendEvent(_ event: NSEvent) {

switch event.type {

case .keyDown:

...

case .keyUp:

...

case .flagsChanged:

...

default:

break

}

super.sendEvent(event)

}

}

​

CGEventTapCreate

[NSEvent addGlobalMonitorForEvents]

CGEventTapCreate

• Pros
• Can receive events for other processes.
• `CGEventTapCreate` can modify the received events.
• But [NSEvent addGlobalMonitorForEvents] cannot modify the received events.
• root permission is not required.
• Cons
• Cannot receive Secure Keyboard Entry.
• Some apps might be broken if other apps observe it.
• Adobe apps will be weird. (e.g., sometimes dropping input events.)
• Note
• User approval is required.
• Enable Accessibility for the app.
• Input Monitoring on macOS >= 10.15.

Demonstration: CGEventTapCreate

cgeventtap-example

( https://github.com/tekezo/osx-event-observer-examples )

​

Code: CGEventTapCreate (1)

CGEventTapExample.m:

​

CGEventMask mask = CGEventMaskBit(kCGEventKeyDown) | ...;

CGEventTapCreate(kCGHIDEventTap, kCGTailAppendEventTap, kCGEventTapOptionListenOnly,

mask, callback, ...);

CGEventTapEnable(tap, true);

​

- (CGEventRef)callback:... {

switch (type) {

case kCGEventKeyDown:

case kCGEventKeyUp:

case kCGEventFlagsChanged:

...

break;

}

return event;

}

​

Code: CGEventTapCreate (2)

CGEventTapExample.m:

​

​

Do not forget to handle `kCGEventTapDisabledByTimeout` in `callback`.

​

​

- (CGEventRef)callback:... {

switch (type) {

case kCGEventTapDisabledByTimeout:

CGEventTapEnable(self.eventTap, true);

Break;

case kCGEventKeyDown:

...

​

Code: CGEventTapCreate (3)

AppDelegate.m:

​

​

You should check Accessibility state, and relaunch app when the state is changed.

(The Accessibility feature is not activated until the app is relaunched.)

​

Note: macOS 10.15 or later, you should also do it for Input Monitoring. (Is API provided ???)

​

​

- (void)applicationDidFinishLaunching:(NSNotification*)notification {

NSDictionary* options = @{(__bridge NSString*)(kAXTrustedCheckOptionPrompt) : @YES};

if (!AXIsProcessTrustedWithOptions((__bridge CFDictionaryRef)options)) {

// Create a timer which calls `AXIsProcessTrusted` and relaunch app if it returns true.

}

}

​

Demonstration: [NSEvent addGlobalMonitorForEvents]

nsevent-example

( https://github.com/tekezo/osx-event-observer-examples )

​

Code: [NSEvent addGlobalMonitorForEvents] (1)

AppDelegate.swift:

​

class AppDelegate: NSObject, NSApplicationDelegate {

func applicationDidFinishLaunching(_: Notification) {

NSEvent.addGlobalMonitorForEvents(

matching: [NSEventMask.keyDown, NSEventMask.keyUp, NSEventMask.flagsChanged],

handler: { (event: NSEvent) in

switch event.type {

case .keyDown:

case .keyUp:

case .flagsChanged:

}})

}

}

​

Code: [NSEvent addGlobalMonitorForEvents] (2)

AppDelegate.swift:

​

​

You should check Accessibility state, and relaunch app when the state is changed.

(The Accessibility feature is not activated until the app is relaunched.)

​

Note: macOS 10.15 or later, you should also do it for Input Monitoring. (Is API provided ???)

​

​

func applicationDidFinishLaunching(_: Notification) {

let options: NSDictionary = [kAXTrustedCheckOptionPrompt.takeRetainedValue() as NSString: true]

if !AXIsProcessTrustedWithOptions(options) {

timer = Timer.scheduledTimer(

withTimeInterval: 3.0,

repeats: true

) { _ in self.relaunchIfProcessTrusted() }

}

}

​

IOHIDQueueRegisterValueAvailableCallback

Demonstration: IOHIDQueueRegisterValueAvailableCallback

iokit-hid-value-example

( https://github.com/tekezo/osx-event-observer-examples )

​

IOKit IOHIDDevice

• Pros
• Can receive events for other processes.
• Can stop event propagation if the device is opened with `kIOHIDOptionsTypeSeizeDevice`.
• root permission is not required.
• Can receive Secure Keyboard Entry if the process is gained root permission.
• Cons
• Cannot receive Apple trackpad events.
• Cannot receive several events (e.g., Internet keys)
• Note
• Can receive events without user approval on macOS <= 10.14.

​

Code: IOHIDQueueRegisterValueAvailableCallback

vendor/include/pqrs/*:

​

You should call `IOHIDQueueStart` before you open the device by `IOHIDDeviceOpen` in order to avoid missing events.

​

IOHIDQueueCreate(...);

IOHIDQueueAddElement(...);

IOHIDQueueRegisterValueAvailableCallback(...);

IOHIDQueueScheduleWithRunLoop(...);

IOHIDQueueStart(...);

IOHIDDeviceOpen(...);

​

​

Others

IOHIDDeviceRegisterInputReportCallback

• Pros
• Similar as IOHIDValue but can get more detailed information via Input Report.
• Cons
• The input report depends on the hardware.�You have to parse HID report descriptor in order to handle the report.�You also have to handle the vendor specific reports. (e.g., fn key on Apple keyboard)

​

Input source

• Pros
• Input source can receive almost events.
• Cons
• User have to choose the input source instead of default one.

​

IOHIKeyboard function pointer replacement

• Pros
• Can receive all events through IOHIKeyboard driver including trackpad events.
• Can receive secure key entries.
• Cons
• kext is required. (Install and load kext requires root permission.)
• This was not formally provided API.�macos >= 10.11 refuses this method.

​

Code: IOHIKeyboard

/IOHIDSystem/IOKit/hidsystem/IOHIKeyboard.h:

​

typedef void (*KeyboardEventAction)(OSObject * target, ...);

typedef void (*KeyboardSpecialEventAction)(OSObject * target, ...);

typedef void (*UpdateEventFlagsAction)(OSObject * target, ...);

​

class IOHIKeyboard : public IOHIDevice

{

protected:

OSObject * _keyboardEventTarget;

KeyboardEventAction _keyboardEventAction;

OSObject * _keyboardSpecialEventTarget;

KeyboardSpecialEventAction _keyboardSpecialEventAction;

OSObject * _updateEventFlagsTarget;

UpdateEventFlagsAction _updateEventFlagsAction;

}

​

Custom Driver

• Pros
• Can receive all hardware events including Internet keys which are dropped in generic keyboard driver.
• Can receive secure key entries.
• Cons
• kext is required. (Might avoid this restriction by using DriverKit since macOS 10.15.)
• Driver is exclusive.�(User cannot use the vendor driver and your driver at the same time.�You have to implement all features which user are required.)

​

Method swizzling + [NSApplication sendEvent]

• Pros
• Can receive any events for other processes which users AppKit.
• Cons
• Obsoleted. Works only on macOS <= 10.5 in the common case.

Appendix

Lifetime of the user approval for Accessibility

• The user approval is effective until the bundle identifier or team identifier are changed.
• The user approval is required each time the binary is changed if the binary is not code signed.
• The user approval state is stored to TCC.db.

​

sqlite3 /Library/Application\ Support/com.apple.TCC/TCC.db

​

sqlite> select * from access;