macOS 10.13 contains a built-in VPN client that natively supports L2TP over IPSec as well as IKEv2.

In this post I describe some parts of the internal architecture of the macOS VPN client. This information will be used in a following article to build an application that replicates some functionalities of the VPN status in the menu bar. This application will also allow to auto connect to an IKEv2 VPN service, something that is currently not possible on macOS.

MarkChart - Mermaid Preview

• Easily preview Mermaid diagrams
• Live update when editing in your preferred editor
• Capture screenshots with customizable margins
• Create PNG from the Terminal
• Free download on the Mac App Store

For Apple employees reading this blog post, you can find my bug report 41950946: scutil doesn't support IKEv2 VPN services here: rdar://41950946.

Please note that I describe undocumented internal APIs of macOS 10.13.5. Apple could change the implementation at any time. The notes below could become outdated and I give no guarantee that it is completely accurate.

Apps to connect to a VPN service

On macOS there are at least 3 built-in applications to connect to a VPN service:

1. Network prefpane in the System Preferences application

2. VPN Status menu in the Menu Bar

3. The command line tool scutil

scutil lets you list your L2TP services with scutil --nc list:

Timac:~ timac$ scutil --nc list
Available network connection services in the current set (*=enabled):
* (Disconnected)   940C2D07-8CF4-492C-A7DE-DD50C337149F PPP --> L2TP       "My_L2TP_VPN" [PPP:L2TP]

It also lets you start the VPN connection:

scutil --nc start My_L2TP_VPN

However this command line tool is not working with IKEv2 VPN services.

VPN Status menu

The VPN Status is implemented as a Menu Extras, a private API unrelated the public NSStatusItem API. It is loaded by the SystemUIServer process and its binary is located at the path /System/Library/CoreServices/Menu Extras/VPN.menu. While the VPN Status app seems like a smaller target than the System Preferences application, I decided instead to analyze how the Network prefpane works. The main reason is that attaching a debugger to the Network prefpane is easier than attaching a debugger to the SystemUIServer process.

System Preferences application

The architecture of the System Preferences application is more complex. This application loads preference panes created using the public PreferencePanes framework /System/Library/Frameworks/PreferencePanes.framework. This plugin architecture is well described in Apple’s documentation Architecture of Preference Panes.

The plugin responsible for the Network preference pane is Network.prefPane.

Network.prefPane

Processes

The Network preference pane is located at the path /System/Library/PreferencePanes/Network.prefPane but (at least part of) its code runs in a separate process using the XPC Contents/XPCServices/com.apple.preference.network.remoteservice.xpc. When the Network preference pane is open, a separate process com.apple.preference.network.remoteservice is visible:

ANPNEService and ANPNEServicesManager

The Network preference pane represents the different services it supports using the Objective-C class ANPTopLevelService. In our case we are really only interested by the ANPNEService subclass.

Below is a stripped down interface for the ANPNEService class. Note that an instance is created using a NEConfiguration object (more about it later) and it has a ne_session_t (more about it later). Also note the methods -connect and -disconnect.

@interface ANPNEService : ANPTopLevelService

@property (retain) NEConfiguration * configuration;
@property (assign) ne_session_t session;

- (instancetype)initWithConfiguration:(NEConfiguration *)inConfiguration;

- (NSString *)name;

- (void)connect;
- (void)disconnect;

@end

These ANPNEService services are managed by the ANPNEServicesManager singleton. Below is a stripped down interface:

@interface ANPNEServicesManager : NSObject

@property (retain) NSArray <ANPNEService*>* mServices;
@property (assign) dispatch_queue_t neServiceQueue;

+ (id)sharedNEServicesManager;

- (void)reloadConfigurations;
- (void)processConfigurations:(NSArray <NEConfiguration*>*)inConfigurations;

@end

The ANPNEService instances are created by the ANPNEServicesManager singleton using the NEConfiguration received from the NetworkExtension.framework:

NetworkExtension.framework

Note that the NetworkExtension.framework (/System/Library/Frameworks/NetworkExtension.framework) is a public framework. However the NEConfigurationManager and NEConfiguration classes are private classes within this public framework.

Here is a simplified interface for NEConfigurationManager:

@interface NEConfigurationManager : NSObject

+ (id)sharedManager;

- (void)loadConfigurationsWithCompletionQueue:(dispatch_queue_t)completionQueue handler:(void (^)(NSArray<NEConfiguration *> * _Nullable configurations, NSError * _Nullable error))handler;

@end

And a simplified interface for NEConfiguration:

@interface NEConfiguration : NSObject

@property (readonly) NSUUID * identifier;
@property (copy) NSString * name;
@property (copy) NEVPN * VPN;

@end

The NEVPN class is also private and relies on the public class NEVPNProtocol:

@interface NEVPN : NSObject

@property (copy) NEVPNProtocol * protocol;

@end

The ne_session_* APIs

Use of ne_session_t in ANPNEService

I previously mentioned that each ANPNEService instance owns a ne_session_t. Let’s look at this.

As we have seen, the designated initializer for ANPNEService takes a NEConfiguration object. The initializer takes care of creating the ne_session_t session. The implementation looks like:

- (instancetype)initWithConfiguration:(NEConfiguration *)inConfiguration
{
    self = [super init];
    if (self)
    {
        _configuration = inConfiguration;
		
        // Get the configuration identifier to initialize the ne_session_t
        NSUUID *uuid = [inConfiguration identifier];
		uuid_t uuidBytes;
		[uuid getUUIDBytes:uuidBytes];
        
        _session = ne_session_create(uuidBytes, NESessionTypeVPN);
        
        [self setupEventCallback];
        [self refreshSession];
    }
    
    return self;
}

The session is then used in the -[ANPNEService connect] method in conjunction with the ne_session_start API:

-(void)connect
{
    ne_session_start([self session]);
}

Similarly the -[ANPNEService disconnect] method uses the session with the ne_session_stop API:

-(void)disconnect
{
    ne_session_stop([self session]);
}

The ne_session_* APIs

The ne_session_* APIs are implemented in the private dynamic library /usr/lib/system/libsystem_networkextension.dylib:

Extracting the ne_session_* APIs is simple using nm:

Timac:~ timac$ nm /usr/lib/system/libsystem_networkextension.dylib | grep "T _ne_session_"
000000000000300d T _ne_session_address_matches_subnets
0000000000004f3f T _ne_session_agent_get_advisory
0000000000004ec2 T _ne_session_agent_get_advisory_interface_index
00000000000012fc T _ne_session_always_on_vpn_configs_present
00000000000012e0 T _ne_session_app_vpn_configs_present
00000000000017ce T _ne_session_cancel
0000000000004ca4 T _ne_session_copy_app_data_from_flow_divert_socket
0000000000004dfa T _ne_session_copy_app_data_from_flow_divert_token
000000000000299e T _ne_session_copy_policy_match
0000000000003a4f T _ne_session_copy_security_session_info
000000000000386d T _ne_session_copy_socket_attributes
0000000000001334 T _ne_session_create
0000000000000f9d T _ne_session_disable_restrictions
0000000000002343 T _ne_session_enable_on_demand
00000000000024a0 T _ne_session_establish_ipc
00000000000010e5 T _ne_session_fallback_advisory
0000000000001189 T _ne_session_fallback_default
0000000000004e5a T _ne_session_get_config_id_from_network_agent
000000000000248d T _ne_session_get_configuration_id
0000000000001d4a T _ne_session_get_info
000000000000194b T _ne_session_get_status
0000000000001d26 T _ne_session_info_type_to_string
000000000000288f T _ne_session_initialize_necp_drop_all
0000000000000d7e T _ne_session_manager_get_pid
0000000000000e0f T _ne_session_manager_has_active_sessions
0000000000000cff T _ne_session_manager_is_running
0000000000001231 T _ne_session_on_demand_configs_present
00000000000043ef T _ne_session_policy_copy_flow_divert_token
0000000000004910 T _ne_session_policy_copy_flow_divert_token_with_key
0000000000002ddf T _ne_session_policy_match_get_filter_unit
0000000000002db5 T _ne_session_policy_match_get_flow_divert_unit
0000000000002e24 T _ne_session_policy_match_get_scoped_interface_index
0000000000002d7a T _ne_session_policy_match_get_service
0000000000002df1 T _ne_session_policy_match_get_service_action
0000000000002d68 T _ne_session_policy_match_get_service_type
0000000000002dca T _ne_session_policy_match_is_drop
0000000000002da0 T _ne_session_policy_match_is_flow_divert
0000000000002e10 T _ne_session_policy_match_service_is_registered
00000000000015b2 T _ne_session_release
000000000000157e T _ne_session_retain
0000000000001df4 T _ne_session_send_barrier
000000000000369f T _ne_session_service_get_dns_service_id
00000000000032ac T _ne_session_service_matches_address
00000000000016ad T _ne_session_set_event_handler
0000000000003748 T _ne_session_set_socket_attributes
0000000000000ef9 T _ne_session_should_disable_nexus
0000000000002337 T _ne_session_start
000000000000210d T _ne_session_start_on_behalf_of
000000000000228e T _ne_session_start_with_options
0000000000003a07 T _ne_session_status_to_string
0000000000002433 T _ne_session_stop
0000000000003c4b T _ne_session_stop_all_with_plugin_type
00000000000051f8 T _ne_session_stop_reason_to_string
0000000000001318 T _ne_session_system_app_vpn_configs_present
0000000000003a2b T _ne_session_type_to_string
0000000000001229 T _ne_session_use_as_system_vpn
0000000000001041 T _ne_session_use_ikev2provider

In our case only a limited number of APIs are really interesting:

extern ne_session_t ne_session_create(uuid_t serviceID, int sessionConfigType);
extern void ne_session_release(ne_session_t session);
extern void ne_session_start(ne_session_t session);
extern void ne_session_stop(ne_session_t session);
extern void ne_session_cancel(ne_session_t session);


typedef void (^ne_session_set_event_handler_block)(xpc_object_t result);
extern void ne_session_set_event_handler(ne_session_t session, dispatch_queue_t queue, ne_session_set_event_handler_block block);


typedef void (^ne_session_get_status_block)(ne_session_status_t result);
extern void ne_session_get_status(ne_session_t session, dispatch_queue_t queue, ne_session_get_status_block block);

Finding the correct prototypes for these functions was simplified thanks to the open source file SCNetworkConnection.c.

The nesessionmanager daemon

If you look at the implementation of the ne_session_* functions, you will note that these functions are sending their request through XPC to the root dameon nesessionmanager located at the path /usr/libexec/nesessionmanager.

This daemon is listening for commands and handles them in the method -(void)[NESMSession handleCommand:fromClient:]. By looking at the logging strings, you can find the code for each command:

cstr_00072C74   "%.30s:%-4d %@: Ignore restart command from %@, a pending start command already exists"
cstr_00072CCA   "%.30s:%-4d %@: Stop current session as requested by an overriding restart command from %@"
cstr_00072D7D   "%.30s:%-4d %@: Received a start command from %@, but start was rejected"
cstr_00072DFD   "%.30s:%-4d %@: Received a start command from %@"
cstr_00072E2D   "%.30s:%-4d %@: Skip a %sstart command from %@: session in state %s"
cstr_00072E73   "%.30s:%-4d %@: Received a stop command from %@ with reason %d"
cstr_00072F7E   "%.30s:%-4d %@: Received an enable on demand command from %@"

For example when an IKEv2 service is started, the method -(void)[NESMIKEv2VPNSession createConnectParametersWithStartMessage:] will be called.

The architecture of the daemon is out of the scope of this article.

Architecture summary

Let’s summarize the VPN architecture from the System Preferences application down to the nesessionmanager root daemon:

VPNStatus: a replacement for the macOS builtin VPN Status menu

In a following article, I will unveil an application called VPNStatus to replicate some functionalities of the VPN Status menu in menu bar:

• list the VPN services and their status
• connect to a VPN service
• disconnect from a VPN service

This application also adds the possibility to:

• auto connect to a VPN service if the application is running
• auto connect to a IKEv2 VPN service, something currently not possible on macOS

References

• Architecture of Preference Panes
• SCNetworkConnection.c

Update (19.07.2018)

The blog post about the VPNStatus application is now available: VPNStatus, a replacement for macOS builtin VPN Status