Mac App Store: Embedding a Command Line tool using paths as arguments

A couple of months ago, I released a new app called Dependencies on the Mac App Store. You can download and try it for free at https://apps.apple.com/app/dependencies/id1538972026.

In this article, I explain how I built the command line support and released it in the Mac App Store. Implementing this feature turned out to be tricky, mostly due to the lack of documentation on this specific subject. This post might be of interest if you are planning to add a Command Line tool to your app distributed on the Mac App Store.

Want to support this blog? Please check out

MarkChart - Mermaid Preview

Easily preview Mermaid diagrams
Live update when editing
Capture screenshots
Create PNG from the Terminal
Free download on the Mac App Store

Dependencies

Dependencies is a developer tool which lets you explore the architecture of your app with an interactive graph by analyzing dSYM files:

You can use the built-in dependencies command line tool to analyze your dSYM from the Terminal. This makes it easy for example to integrate Dependencies with your CI.

In this post I will detail the difficulties to implement this feature and how I solved them:

😌 Embedding the command line tool
🥵 Adopting the App Sandbox
😣 Creating the symlink /usr/local/bin/dependencies
😮‍💨 Creating a simple UI
😫 Dealing with the Mac App Store rejections

Some notes about the code snippets:

You can reuse this code under the MIT license
The code is written in Objective-C
You can find the complete source code on https://github.com/Timac/Mac-App-Store-Embedding-a-Command-Line-tool-using-paths-as-arguments

Embedding the command line tool

Copying the command line tool into the main app is not the most complicated part. But you still need to be careful to make sure that the app is properly code signed.

Copying

I created in the Xcode project a new target called DependenciesCLI which compiles the dependencies Command Line tool. I decided to embed the tool at Dependencies.app/Contents/MacOS/CLI/dependencies.

Copying the Command Line tool into the main app is done using a copy build phase:

Signing & Capabilities

Since the app will be released on the Mac App Store, don’t forget to enable the App Sandbox in the Signing & Capabilities. The Hardened Runtime is required in order to notarize your app, for example to distribute a beta version with Developer ID.

Embedded Info.plist

To be properly code signed, the Info.plist needs to be embedded into the binary. This can be done using the CREATE_INFOPLIST_SECTION_IN_BINARY setting:

Adopting the App Sandbox

At this point, the Command Line tool is embedded into the main app and properly code signed. Now comes the tricky part. The dependencies Command Line tool takes some paths as parameters but needs to adopt the App Sandbox:

dependencies -i /Users/timac/MyiOSApp.dSYM -o /Users/timac/MyiOSApp.html

Accessing random files is evidently blocked by the App Sandbox. Also note that the entitlements com.apple.security.files.user-selected.read-only and com.apple.security.files.user-selected.read-write only allow your app to access files selected by the user using an Open dialog. This entitlement doesn’t let you access files passed as parameter.

In order to get access to random files outside of the app container, you can ask the user to select a folder using an Open dialog. The app will then be allowed to access any file inside this selected folder… as long as the app is running. If the app is quit and relaunched (as you might expect from a command line tool), you will need to use Security-Scoped Bookmarks to keep this access persistent.

Security-Scoped Bookmarks

The Security-Scoped Bookmarks are fairly well documented in the App Sandbox Design Guide. Here is a summary on how the Security-Scoped Bookmarks works:

The app can ask the user via an Open dialog for access to a folder outside of its container. However this access does not automatically persist across app launches.
The app can create an app-scoped bookmark to keep a persistent access to this user-specified file or folder. Note that the app needs to contain the entitlement com.apple.security.files.bookmarks.app-scope according to the documentation.
This bookmark can be saved for example in the app preferences via NSUserDefaults.
When the app is launched and it needs to access the file or folder, it can resolve the app security-scoped bookmark.

Here is an Objective-C implementation to create and resolve security-scoped bookmark. Note that you can find this source code on GitHub:

@implementation DEPScopeBookmark

+(NSData *)createSecurityScopeBookmarkDataForURL:(NSURL *)inURL
{
	if(inURL != nil)
	{
		NSError *error = nil;
		NSData *bookmarkData = [inURL bookmarkDataWithOptions:NSURLBookmarkCreationWithSecurityScope includingResourceValuesForKeys:nil relativeToURL:nil error:&error];
		if(bookmarkData != nil)
		{
			return bookmarkData;
		}
		else
		{
			NSLog(@"Could not create the Bookmark data for %@ due to %@", [inURL path], error);
		}
	}
	
	return nil;
}

+(NSURL *)resolveBookmarkData:(NSData **)inOutBookmarkData
{
	if(inOutBookmarkData == nil)
		return nil;
	
	NSURL *outBookmarkURL = nil;
	
	NSError *error = nil;
	NSData *bookmarkData = *inOutBookmarkData;
	if (bookmarkData != nil)
	{
		BOOL bookmarkDataIsStale = NO;
		NSURL *theURL = [NSURL URLByResolvingBookmarkData:bookmarkData options:NSURLBookmarkResolutionWithSecurityScope relativeToURL:nil bookmarkDataIsStale:&bookmarkDataIsStale error:&error];
		if (theURL != nil && bookmarkDataIsStale)
		{
			// Update the bookmark data
			NSData *updatedBookmark = [DEPScopeBookmark createSecurityScopeBookmarkDataForURL:theURL];
			if(updatedBookmark != nil)
			{
				*inOutBookmarkData = updatedBookmark;
			}
		}
		
		if(theURL != nil)
		{
			BOOL startAccessingSecurityScopedResourceResult = [theURL startAccessingSecurityScopedResource];
			if(!startAccessingSecurityScopedResourceResult)
			{
				NSLog(@"Could not start accessing the scoped URL");
			}
			else
			{
				outBookmarkURL = theURL;
			}
		}
		else
		{
			if(error != nil)
			{
				NSLog(@"Could not get the url for the bookmark data due to %@", error);
			}
			else
			{
				NSLog(@"Could not get the url for the bookmark data");
			}
		}
	}
	else
	{
		NSLog(@"No Bookmark data");
	}
	
	return outBookmarkURL;
}

@end

It all sounds simple… until you realize that a command line tool can’t prompt the user to select a folder using an Open dialog. The solution I came up with, is to prompt the user in the main app and pass the Security-Scoped Bookmark to the command line, easy right?

App Groups

Since the app is sandboxed, inter-process communication is restricted. To share data between multiple apps (from the same developer), an App Group is needed. This feature can be enabled by adding the entitlement com.apple.security.application-groups in both the app and the command line tool. A group container is then available to share data.

Here is a screenshot showing the content of the entitlements file of the main app. Note the value $(TeamIdentifierPrefix)app.dependencies.dependencies.preferences inside the com.apple.security.application-groups array to create a group container:

And here are the similar entitlements for the command line tool:

With the group container, the app can now create the app-scoped bookmark and save it to the group container. Here is how the App Group is created in Dependencies:

// The App Group to share the preferences between the QuickLook plugin, CLI and the app
NSString *const kPreferencesAppGroup = @"QFL3YR6JR6.app.dependencies.dependencies.preferences";


@implementation DEPPreferences

+(NSUserDefaults*)sharedAppGroupPreferencesSuite
{
	static NSUserDefaults *sUserDefault = nil;
	if(sUserDefault == nil)
	{
		sUserDefault = [[NSUserDefaults alloc] initWithSuiteName:kPreferencesAppGroup];
	}
	
	return sUserDefault;
}

@end

And how an app-scoped bookmark can be stored to and resolved from the NSUserDefaults:

// The preference where the Scope Bookmark is saved to enable the command line support
NSString* const kCommandLineSupportSelectedFolder = @"CommandLineSupportSelectedFolder";

@implementation DEPPreferences (Bookmark)

+(NSURL *)resolveBookmarkDataForKey:(NSString *)inPrefKey inUserDefaults:(NSUserDefaults *)inUserDefaults
{
	NSURL *outBookmarkURL = nil;
	if(inPrefKey != nil && inUserDefaults != nil)
	{
		NSData *bookmarkData = [inUserDefaults objectForKey:inPrefKey];
		if (bookmarkData != nil)
		{
			NSData *updatedBookmarkData = bookmarkData;
			outBookmarkURL = [DEPScopeBookmark resolveBookmarkData:&updatedBookmarkData];
			if(updatedBookmarkData != bookmarkData)
			{
				[inUserDefaults setObject:updatedBookmarkData forKey:inPrefKey];
			}
		}
		else
		{
			NSLog(@"No Bookmark data stored for %@", inPrefKey);
		}
	}
	
	return outBookmarkURL;
}

+(void)createSecurityScopeCommandLineBookmark:(NSURL *)inURL
{
	if(inURL != nil)
	{
		NSData *bookmark = [DEPScopeBookmark createSecurityScopeBookmarkDataForURL:inURL];
		if(bookmark != nil)
		{
			[[NSUserDefaults standardUserDefaults] setObject:bookmark forKey:kCommandLineSupportSelectedFolder];
		}
	}
}

+(NSURL *)resolveSecurityScopeCommandLineBookmark
{
	return [DEPPreferences resolveBookmarkDataForKey:kCommandLineSupportSelectedFolder inUserDefaults:[NSUserDefaults standardUserDefaults]];
}

In theory the command line tool could then use the app-scoped bookmark… But not so fast: Security-Scoped Bookmarks are signed and can only be resolved by the app that created it. So our command line tool can’t use an app-scoped bookmark created by the main app and shared by App Group…

Non-secure bookmark

At this point, I got stuck for several days. I read multiple times the whole App Sandbox documentation with no solution in sight… until I noticed an interesting not(e) in the Enabling App Sandbox Inheritance documentation:

A child process can’t access the files opened by the main app, unless a bookmark is passed to the child process. Nothing new until you read this sentence:

The bookmark need not be a security-scoped bookmark, but it can be, if desired.

What does it tell us? The main app can pass non-secure bookmark to a child process. How does it help in our case where the command line tool is not a child process? Well it turns out that the main app can pass the non-secure bookmark to the command line tool as long as both binaries are running at the same time.

Here is how Dependencies creates and resolves a non-secure bookmark to the shared App Group preferences:

@implementation DEPPreferences (Bookmark)

+(void)createSharedCommandLineBookmark:(NSURL *)inURL
{
	if(inURL != nil)
	{
		NSError *error = nil;
		NSData *bookmarkData = [inURL bookmarkDataWithOptions:NSURLBookmarkCreationMinimalBookmark includingResourceValuesForKeys:nil relativeToURL:nil error:&error];
		if(bookmarkData != nil)
		{
			[[DEPPreferences sharedAppGroupPreferencesSuite] setObject:bookmarkData forKey:kCommandLineSupportSelectedFolder];
		}
		else
		{
			NSLog(@"Could not create the command line bookmark data for %@ due to %@", [inURL path], error);
		}
	}
}

+(NSURL *)resolveSharedCommandLineBookmark
{
	NSData *bookmarkData = [[DEPPreferences sharedAppGroupPreferencesSuite] objectForKey:kCommandLineSupportSelectedFolder];
	if(bookmarkData != nil)
	{
		BOOL bookmarkDataIsStale = NO;
		NSError *error = nil;
		NSURL *theURL = [NSURL URLByResolvingBookmarkData:bookmarkData options:NSURLBookmarkResolutionWithoutUI relativeToURL:nil bookmarkDataIsStale:&bookmarkDataIsStale error:&error];
		
		if(theURL != nil)
		{
			return theURL;
		}
		else if(error != nil)
		{
			NSLog(@"Could not get the url for the command line bookmark data due to %@", error);
		}
		else
		{
			NSLog(@"Could not get the url for the command line bookmark data");
		}
	}
	
	return nil;
}

@end

Note the use of NSURLBookmarkCreationMinimalBookmark instead of NSURLBookmarkCreationWithSecurityScope when creating the bookmark.

Whole flow

We now solved all the technical problems and the command line tool can get access to the files passed as parameter by following these steps:

The user is prompted to select a folder in the main app (for example ~)

@implementation DEPPreferencesCommandLineViewController

-(IBAction)doEnableCommandLine:(id)sender
{
	NSOpenPanel *openPanel = [NSOpenPanel openPanel];
	[openPanel setAllowsMultipleSelection:NO];
	[openPanel setCanChooseFiles:NO];
	[openPanel setCanChooseDirectories:YES];
	[openPanel setResolvesAliases:YES];
	[openPanel setDirectoryURL:[NSURL fileURLWithPath:[@"~" stringByExpandingTildeInPath]]];
	[openPanel setPrompt:@"Select"];
	[openPanel setMessage:@"Select a folder that Dependencies will be able to access from the command line."];
	
	NSModalResponse response = [openPanel runModal];
	if(response == NSModalResponseOK)
	{
		NSURL *url = [openPanel URLs].firstObject;
		if(url != nil)
		{
			[DEPPreferences createSharedCommandLineBookmark:url];
			[DEPPreferences createSecurityScopeCommandLineBookmark:url];
			self.bookmarkURL = [DEPPreferences resolveSecurityScopeCommandLineBookmark];
			[self updateStatus];
		}
	}
}

@end

A non-secure bookmark is saved to the shared group container

[DEPPreferences createSharedCommandLineBookmark:url];

The user is asked to manually launch the command line tool while the main app is still running
The command line tool reads the shared non-secure bookmark

NSURL *bookmarkURL = [DEPPreferences resolveSharedCommandLineBookmark];

The command line tool creates a security scope bookmark from the non-secure bookmark in order to get persistent access to the folder

int main(int argc, const char * argv[])
{
	@autoreleasepool
	{
		NSString *commandLineBookmarkPath = nil;
		
		// If there is a shared bookmark, try to convert it to a security scope bookmark
		NSURL *bookmarkURL = [DEPPreferences resolveSharedCommandLineBookmark];
		if(bookmarkURL != nil)
		{
			[DEPPreferences createSecurityScopeCommandLineBookmark:bookmarkURL];
			bookmarkURL = [DEPPreferences resolveSecurityScopeCommandLineBookmark];
			if(bookmarkURL != nil)
			{
				//NSLog(@"Successfully imported the shared bookmark for %@", bookmarkURL);
				commandLineBookmarkPath = [bookmarkURL path];
			}
			
			[DEPPreferences removeSharedCommandLineBookmark];
		}
		
		// Try to load the security scope bookmark
		NSURL *securityScopeBookmarkURL = [DEPPreferences resolveSecurityScopeCommandLineBookmark];
		if(securityScopeBookmarkURL != nil)
		{
			//NSLog(@"Command line support enabled for %@", securityScopeBookmarkURL);
			commandLineBookmarkPath = [securityScopeBookmarkURL path];
		}
		
		if(commandLineBookmarkPath == nil)
		{
			NSLog(@"Please enable the command line support in the Dependencies preferences.");
			return 1;
		}
		
		[...]
	}
	
	return 0;
}

The command line tool has now persistent access to the folder even if the computer is restarted. It can access the files passed as parameters as long as they are inside the user-specified folder 🚀

Creating the symlink `/usr/local/bin/dependencies`

The command line tool can access the files passed as parameter but there is one improvement possible: at the moment you need to execute the command line with the full path /Applications/Dependencies.app/Contents/MacOS/CLI/dependencies instead of just dependencies. In this section, I discuss how to create a symlink /usr/local/bin/dependencies pointing to /Applications/Dependencies.app/Contents/MacOS/CLI/dependencies.

createSymbolicLinkAtURL:withDestinationURL:error:

My first idea was to simply use -[NSFileManager createSymbolicLinkAtURL:withDestinationURL:error:] to create the symlink:

NSURL *destinationURL = [NSURL fileURLWithPath:@"/Applications/Dependencies.app/Contents/MacOS/CLI/dependencies"];
NSURL *symlinkURL = [NSURL fileURLWithPath:@"/usr/local/bin/dependencies"];
NSError* symlinkError = nil;
[[NSFileManager defaultManager] createSymbolicLinkAtURL:symlinkURL withDestinationURL:destinationURL error:&symlinkError];
NSLog(@"symlinkError: %@", symlinkError);

But there are 2 problems with this approach:

Since /usr/local/bin/ doesn’t exist on a clean macOS installation, the permissions of this folder could be anything depending on how the folder was created.
You can’t create a symlink from a sandboxed app unless you have the NSWorkspaceAuthorizationType entitlement

NSWorkspaceAuthorizationType entitlement

Sandboxed apps distributed on the Mac App Store can request a special com.apple.developer.security.privileged-file-operations entitlement to perform privileged file operations at https://developer.apple.com/contact/request/privileged-file-operations/. The request form:

Once you get the entitlement, you should be able to create symlinks according to the NSWorkspaceAuthorizationType documentation by prompting the user using NSWorkspaceAuthorizationTypeCreateSymbolicLink with some code like:

[[NSWorkspace sharedWorkspace] requestAuthorizationOfType:NSWorkspaceAuthorizationTypeCreateSymbolicLink completionHandler:^(NSWorkspaceAuthorization *theAuth, NSError *theError)
{
	if(theError != nil)
	{
		NSLog(@"requestAuthorizationOfType returned %@", theError);
		return;
	}
	
	if(theAuth != nil)
	{
		NSFileManager *fileManager = [NSFileManager fileManagerWithAuthorization:theAuth];
		NSError *symlinkError = nil;
		[fileManager createSymbolicLinkAtPath:@"/usr/local/bin/dependencies" withDestinationPath:@"/Applications/Dependencies.app/Contents/MacOS/CLI/dependencies" error:&symlinkError];
		if(symlinkError != nil)
		{
			NSLog(@"createSymbolicLinkAtPath returned %@", symlinkError);
		}
	}
	else
	{
		NSLog(@"requestAuthorizationOfType returned no NSWorkspaceAuthorization");
	}
}];

Implementation in Dependencies

I ended up not requesting this entitlement for Dependencies for several reasons:

Using dependencies instead of the full path is a small Quality of Life improvement. This entitlement is not required for my use case.
Dependencies is a developer tool. Its users most likely know how to manually create a symlink in the Terminal.
I didn’t want to wait for the entitlement approval/rejection before the 1.0 release.
Apple clearly states that this is a temporary entitlement and API. I didn’t want to rely on something that might go away at any time.

I decided to simply explain how to manually create the symlink in the UI.

Creating a simple UI

With all the technical pieces put together, I tried to design a clear UI to enable the command line support:

The user is asked to follow 3 steps:

Select a folder to give persistent access to the command line tool
Run the command line tool in the Terminal while the main app is still running
Optionally create the symlink

Dealing with the Mac App Store rejections

At the time of submission to the App Store, I was concerned by 2 features which could possibly lead to a rejection.

QuickLook

Due to a WebKit regression introduced in macOS Big Sur, the QuickLook integration broke. The only workaround I could find was to set an obscure com.apple.security.temporary-exception.mach-lookup.global-name entitlement. I clearly indicated this information (and the radar number) in the notes for the reviewers. So far this never caused a rejection 🤞

Command Line support

As you saw in this blog post, embedding a Command Line tool using paths as arguments is not simple. Nonetheless the initial version 1.0 was approved 🥳

That’s only when I submitted the version 1.2 for review, that the app got rejected for this feature:

Guideline 2.3 - Performance


Your app does not achieve the core functionality described in your marketing materials or release notes.

Preferences > Command Line > Grant access to Macintosh HD

Specifically,

We advise having the open dialog preset to the Home folder, allowing the user to select lower levels of the filesystem if they desire to.

The user is provided the option to select this functionality within the app. Only after attempting to use this functionality is the user told that it is not available without an additional install. It will be necessary to remove this functionality from the app.

The first reason of rejection was clear: when the user was prompted to select a folder, the default folder was /. This could lead to security concerns. To resolve this issue, I changed the default folder selected in the NSOpenPanel to ~:

[openPanel setDirectoryURL:[NSURL fileURLWithPath:@"/"]];
->
[openPanel setDirectoryURL:[NSURL fileURLWithPath:[@"~" stringByExpandingTildeInPath]]];

On the other hand, the second reason for the rejection was obscure. My guess is that the reviewer expected the users to need to download an external tool to use the command line support. I resolved this issue by explaining the feature in the Resolution Center. The reviewer accepted the next submitted version 😘

Want to support this blog? Please check out

DotChart - DOT and Graphviz Preview

Easily preview DOT diagrams
Live update when editing
Capture screenshots
Quick Look integration
Free download on the Mac App Store

Conclusion

Releasing an app on the Mac App Store with an embedded Command Line tool using paths as arguments turned out to be much more tricky than I expected. This is however possible and you can check it out with Dependencies.

I hope that this post will be useful to other Mac developers trying to achieve a similar feature. I open-sourced the relevant parts of Dependencies dealing with this feature on https://github.com/Timac/Mac-App-Store-Embedding-a-Command-Line-tool-using-paths-as-arguments.

Update 16.05.2021:

Jeff Johnson pointed up 2 inaccuracies in a tweet:

The Hardened Runtime is currently not mandatory to release an app on the Mac App Store. It is however required in order to notarize your app, for example to distribute a beta version with Developer ID.
Although the documentation states that the com.apple.security.files.bookmarks.app-scope entitlement is needed to enable Security-Scoped Bookmarks, it seems not to be required in practice.

Dependencies

Embedding the command line tool

Copying

Signing & Capabilities

Embedded Info.plist

Adopting the App Sandbox

Security-Scoped Bookmarks

App Groups

Non-secure bookmark

Whole flow

Creating the symlink /usr/local/bin/dependencies

createSymbolicLinkAtURL:withDestinationURL:error:

NSWorkspaceAuthorizationType entitlement

Implementation in Dependencies

Creating a simple UI

Dealing with the Mac App Store rejections

QuickLook

Command Line support

Conclusion

Creating the symlink `/usr/local/bin/dependencies`