Netatalk Manual

Setting up Netatalk

Setting up the AFP file server

AFP (the Apple Filing Protocol) is a protocol used on the Apple Macintosh for file services. The protocol has evolved over the years. The final revision of the protocol, AFP 3.4, was introduced with OS X Lion (10.7).

Netatalk’s afpd daemon offers AFP fileservices to Apple clients. The configuration is managed through the afp.conf file which uses an ini style configuration syntax.

Netatalk provides compatibility with Time Machine for remote backups, and Spotlight for indexed searching.

To make a volume a Time Machine target, use the volume option time machine = yes.

To enable Spotlight indexing globally or for a volume, set the option spotlight = yes where appropriate.

Starting with Netatalk 2.1, UNIX symlinks can be used on the server. Semantics are the same as for e.g. NFS, i.e. they are not resolved on the server side but instead it’s completely up to the client to resolve them, resulting in links that point somewhere inside the client’s filesystem view.

afp.conf

afp.conf is the configuration file used by afpd to determine the behaviour and configuration of the AFP file server and the AFP volume that it provides.

The afp.conf is divided into several sections:

[Global]

The global section defines general server options

[Homes]

The homes section defines user home volumes

Any section not called Global or Homes is interpreted as an AFP volume.

For sharing user homes by defining a Homes section you must specify the option basedir regex which can be a simple string with the path to the parent directory of all user homes or a regular expression.

Example:

[Homes]
basedir regex = /home

Now any user logging into the AFP server will have a user volume available whose path is /home/NAME.

A more complex setup would be a server with a large amount of user homes which are split across e.g. two different filesystems:

The following configuration is required:

[Homes]
basedir regex = /RAID./.*homes

If basedir regex contains a symlink, set the canonicalized absolute path. When /home links to /usr/home:

[Homes]
basedir regex = /usr/home

For a more detailed explanation of the available options, please refer to the afp.conf man page.

CNID backends

Unlike other protocols like SMB or NFS, the AFP protocol mostly refers to files and directories by ID and not by a path. These IDs are called CNID, which stand for Catalog Node ID. A typical AFP request uses a directory ID and a filename, something like “server, please open the file named ‘Test’ in the directory with id 167”. For example “Aliases” on the Mac basically work by ID (with a fallback to the absolute path in more recent AFP clients. But this applies only to Finder, not to applications).

Every file in an AFP volume has to have a unique file ID. CNIDs must, according to the AFP specification, never be reused. The IDs are represented as 32 bit numbers, and directory IDs use the same ID pool. So, after ~4 billion files/folders have been written to an AFP volume, the ID pool is depleted and no new file can be written to the volume. Some of Netatalk’s CNID backends may attempt to reuse available IDs after depletion, which is technically in violation of the spec, but may enable continuous use on long-lived volumes.

Netatalk needs to map IDs to files and folders in the host filesystem. To achieve this, several different CNID backends are available and can be selected with the cnid scheme option in the afp.conf configuration file. A CNID backend is basically a database storing ID <-> name mappings.

For the CNID backends which use a zero-configuration database, the database files are by default located in a netatalk/CNID subdirectory of your system’s state directory path, e.g. /var/lib. You can change the state directory path with -Dwith-statedir-path=PATH at compile time.

There is a command line utility called dbd available which can be used to verify, repair and rebuild the CNID database.

NOTE: There are some CNID related things you should keep in mind when working with netatalk:

Below follows descriptions of the various CNID backends that are included with netatalk. You can choose to build one or several of them at compile time. Run the command afpd -v to see which backends are available to you, as well as which one is the default.

dbd

The “Database Daemon” backend is built on Berkeley DB. Access to the CNID database is restricted to the cnid_dbd daemon process. afpd processes communicate with the cnid_dbd daemon for database reads and updates, which is in turn launched and controlled by the cnid_metad daemon.

This is the most reliable and proven backend for daily use.

sqlite

A performant and lean CNID database backend that uses the SQLite v3 embedded database engine.

mysql

CNID backend using a MySQL server. The MySQL server has to be provisioned by the system administrator, and Netatalk configured to connect to it over the network.

See afp.conf(5) for documentation of the configuration options.

last

The last backend is an in-memory Trivial Database (tdb). It is not persistent, with IDs valid only for the current session. Starting with netatalk 3.0, it operates in read only mode. This backend is useful e.g. for mounting CD-ROMs, or for automated testing.

This is basically equivalent to how afpd stored CNID data in netatalk 1.5 and earlier.

Charsets/Unicode

Why Unicode?

Internally, computers don’t know anything about characters and texts, they only know numbers. Therefore, each letter is assigned a number. A character set, often referred to as charset or codepage, defines the mappings between numbers and letters.

If two or more computer systems need to communicate with each other, the have to use the same character set. In the 1960s the ASCII (American Standard Code for Information Interchange) character set was defined by the American Standards Association. The original form of ASCII represented 128 characters, more than enough to cover the English alphabet and numerals. Up to date, ASCII has been the normative character scheme used by computers.

Later versions defined 256 characters to produce a more international fluency and to include some slightly esoteric graphical characters. Using this mode of encoding each character takes exactly one byte. Obviously, 256 characters still wasn’t enough to map all the characters used in the various languages into one character set.

As a result localized character sets were defined later, e.g the ISO-8859 character sets. Most operating system vendors introduced their own characters sets to satisfy their needs, e.g. IBM defined the codepage 437 (DOSLatinUS), Apple introduced the MacRoman codepage and so on. The characters that were assigned number larger than 127 were referred to as extended characters. These character sets conflict with another, as they use the same number for different characters, or vice versa.

Almost all of those characters sets defined 256 characters, where the first 128 (0-127) character mappings are identical to ASCII. As a result, communication between systems using different codepages was effectively limited to the ASCII charset.

To solve this problem new, larger character sets were defined. To make room for more character mappings, these character sets use at least 2 bytes to store a character. They are therefore referred to as multibyte character sets.

One standardized multibyte charset encoding scheme is known as Unicode. A big advantage of using a multibyte charset is that you only need one. There is no need to make sure two computers use the same charset when they are communicating.

Character sets used by Apple

In the past, Apple clients used single-byte charsets to communicate over the network. Over the years Apple defined a number of codepages, western users will most likely be using the MacRoman codepage.

Codepages defined by Apple include:

Starting with Mac OS X and AFP3, UTF-8 is used. UTF-8 encodes Unicode characters in an ASCII compatible way, each Unicode character is encoded into 1-6 ASCII characters. UTF-8 is therefore not really a charset itself, it’s an encoding of the Unicode charset.

To complicate things, Unicode defines several normalization forms. While samba uses precomposed Unicode, which most UNIX tools prefer as well, Apple decided to use the decomposed normalization.

For example lets take the character ‘ä’ (lowercase a with diaeresis). Using the precomposed normalization, Unicode maps this character to 0xE4. In decomposed normalization, ‘ä’ is actually mapped to two characters, 0x61 is the mapping for an ‘a’, 0x308 is the mapping for a COMBINING DIAERESIS.

Netatalk refers to precomposed UTF-8 as UTF8 and to decomposed UTF-8 as UTF8-MAC.

afpd and character sets

To support new AFP 3.x and older AFP 2.x clients at the same time, afpd needs to be able to convert between the various charsets used. AFP 3.x clients always use UTF8-MAC, while AFP 2.x clients use one of the Apple codepages.

At the time of writing, netatalk supports the following Apple codepages:

afpd handles three different character set options:

unix charset

This is the codepage used internally by your operating system. If not specified, it defaults to UTF8. If LOCALE is specified and your system support UNIX locales, afpd tries to detect the codepage. afpd uses this codepage to read its configuration files, so you can use extended characters for volume names, login messages, etc.

mac charset

As already mentioned, older Mac OS clients (up to AFP 2.2) use codepages to communicate with afpd. However, there is no support for negotiating the codepage used by the client in the AFP protocol. If not specified otherwise, afpd assumes the MacRoman codepage is used. In case your clients use another codepage, e.g. MacCyrillic, you’ll have to explicitly configure this.

vol charset

This defines the charset afpd should use for filenames on disk. By default, it is the same as unix charset. If you have iconv installed, you can use any iconv provided charset as well.

afpd needs a way to preserve extended Macintosh characters, or characters illegal in UNIX filenames, when saving files on a UNIX filesystem. Earlier versions used the the so called CAP encoding. An extended character (>0x7F) would be converted to a :xx hex sequence, e.g. the Apple Logo (MacRoman: 0xF0) was saved as :f0. Some special characters will be converted as to :xx notation as well. ‘/’ will be encoded to :2f, if usedots was not specified, a leading dot ‘.’ will be encoded as :2e.

Even though this version now uses UTF8 as the default encoding for filenames, ‘/’ will be converted to ‘:’. For western users another useful setting could be vol charset = ISO-8859-15.

If a character cannot be converted from the mac charset to the selected vol charset, you’ll receive a -50 error on the Mac. Note: Whenever you can, please stick with the default UTF8 volume format.

Authentication

AFP authentication basics

Apple chose a flexible model called “User Authentication Modules” (UAMs) for authentication purposes between AFP client and server. An AFP client initially connecting to an AFP server will ask for the list of UAMs which the server provides, and will choose the one with strongest encryption that the client supports.

Several UAMs have been developed by Apple over the time, some by 3rd-party developers.

UAMs supported by Netatalk

Netatalk supports the following ones by default:

There exist other optional UAMs as well:

You can configure which UAMs should be activated by defining “uam list” in Global section. afpd will log which UAMs it’s using and if problems occur while activating them in either netatalk.log or syslog at startup time. asip-status can be used to query the available UAMs of AFP servers as well.

Having a specific UAM available at the server does not automatically mean that a client can use it. Client-side support is also necessary. For older Macintoshes running Classic Mac OS, DHCAST128 support exists since AppleShare client 3.8.x.

On macOS, there exist some client-side techniques to make the AFP-client more verbose, so one can have a look at what’s happening while negotiating the UAMs to use. Compare with this hint.

Which UAMs to activate?

It depends primarily on your needs and on the kind of macOS clients you have to support. If your network consists of exclusively macOS (Mac OS X) clients, DHX2 is sufficient, and provides the strongest encryption.

For a more detailed overview over the technical implications of the different UAMs, please have a look at Apple’s File Server Security pages.

Using different authentication sources with specific UAMs

Some UAMs provide the ability to use different authentication “backends”, namely uams_clrtxt.so, uams_dhx.so and uams_dhx2.so. They can use either classic UNIX passwords from /etc/passwd (/etc/shadow) or PAM if the system supports that. uams_clrtxt.so can be symlinked to either uams_passwd.so or uams_pam.so, uams_dhx.so to uams_dhx_passwd.so or uams_dhx_pam.so and uams_dhx2.so to uams_dhx2_passwd.so or uams_dhx2_pam.so.

So, if it looks like this in Netatalk’s UAMs folder (per default /etc/netatalk/uams/) then you’re using PAM, otherwise classic UNIX passwords.

uams_clrtxt.so -> uams_pam.so
uams_dhx.so -> uams_dhx_pam.so
uams_dhx2.so -> uams_dhx2_pam.so

The main advantage of using PAM is that one can integrate Netatalk in centralized authentication scenarios, e.g. via LDAP, NIS and the like. Please always keep in mind that the protection of your user’s login credentials in such scenarios also depends on the strength of encryption that the UAM in question supplies. So think about eliminating weak UAMs like “ClearTxt Passwrd” and “Randnum exchange” completely from your network.

Netatalk UAM overview table

A small overview of the officially supported UAMs.

UAM No User Auth Cleartxt Passwrd RandNum Exchange DHCAST128 DHX2 Kerberos V
Password length guest access max 8 chars max 8 chars max 64 chars max 256 chars Kerberos tickets
Client support built-in into all Mac OS versions built-in in all Mac OS versions except 10.0. Has to be activated explicitly in later Mac OS X versions built-in into almost all Mac OS versions built-in since AppleShare client 3.8.4, available as a plug-in for 3.8.3, integrated in macOS’s AFP client built-in since Mac OS X 10.2 built-in since Mac OS X 10.2
Encryption Enables guest access without authentication between client and server. Password will be sent in cleartext over the wire. Just as bad as it sounds, therefore avoid at all if possible (note: providing NetBoot services requires the ClearTxt UAM) 8-byte random numbers are sent over the wire, comparable with DES, 56 bits. Vulnerable to offline dictionary attack. Requires passwords in clear on the server. Password will be encrypted with 128 bit CAST, user will be authenticated against the server but not vice versa. Therefore weak against man-in-the-middle attacks. Password will be encrypted with 128 bit CAST in CBC mode. User will be authenticated against the server but not vice versa. Therefore weak against man-in-the-middle attacks. Password is not sent over the network. Due to the service principal detection method, this authentication method is vulnerable to man-in-the-middle attacks.
Server support uams_guest.so uams_cleartxt.so uams_randnum.so uams_dhx.so uams_dhx2.so uams_gss.so
Password storage None Either system auth or PAM Passwords stored in clear text in a separate text file Either system auth or PAM Either system auth or PAM At the Kerberos Key Distribution Center

ACL Support

ACL support for AFP is implemented for ZFS ACLs on Solaris and derived platforms and for POSIX 1e ACLs on Linux.

Configuration

For a basic mode of operation there’s nothing to configure. Netatalk reads ACLs on the fly and calculates effective permissions which are then send to the AFP client via the so called UARights permission bits. On a Mac, the Finder uses these bits to adjust permission in Finder windows. Example: a folder whose UNIX mode is read-only and an ACL giving the user write access, will display the effective read-write permission. Without the permission mapping the Finder would display a read-only icon and the user wouldn’t be able to write to the folder.

By default, the effective permission of the authenticated user are only mapped to the mentioned UARightspermission structure, not the UNIX mode. You can adjust this behaviour with the configuration option map acls.

However, neither in Finder “Get Info” windows nor in the Terminal will you be able to see the ACLs, because of how ACLs in macOS are designed. If you want to be able to display ACLs on the client, things get more involved as you must then setup both client and server to be part on a authentication domain (directory service, e.g. LDAP, OpenDirectory). The reason is, that in macOS ACLs are bound to UUIDs, not just uid’s or gid’s. Therefore afpd must be able to map every filesystem uid and gid to a UUID so that it can return the server side ACLs which are bound to UNIX uid and gid mapped to macOS UUIDs.

Netatalk can query a directory server using LDAP queries. Either the directory server already provides an UUID attribute for user and groups (Active Directory, Open Directory) or you reuse an unused attribute (or add a new one) to you directory server (e.g. OpenLDAP).

In detail:

  1. For Solaris/ZFS: ZFS Volumes

    You should configure a ZFS ACL know for any volume you want to use with Netatalk:

     aclinherit = passthrough
 aclmode = passthrough

    For an explanation of what this knob does and how to apply it, check your hosts ZFS documentation (e.g. man zfs).

  2. Authentication Domain

    Your server and the clients must be part of a security association where identity data is coming from a common source. ACLs in Darwin are based on UUIDs and so is the ACL specification in AFP 3.2. Therefore your source of identity data has to provide an attribute for every user and group where a UUID is stored as a ASCII string. In other words:

macOS ACLs

With Access Control Lists (ACLs), macOS offers a powerful extension of the traditional UNIX permissions model. An ACL is an ordered list of Access Control Entries (ACEs) explicitly granting or denying a set of permissions to a given user or group.

Unlike UNIX permissions, which are bound to user or group IDs, ACLs are tied to UUIDs. For this reason accessing an object’s ACL requires server and client to use a common directory service which translates between UUIDs and user/group IDs.

ACLs and UNIX permissions interact in a rather simple way. As ACLs are optional UNIX permissions act as a default mechanism for access control. Changing an objects’s UNIX permissions will leave its ACL intact and modifying an ACL will never change the object’s UNIX permissions. While doing access checks, macOS first examines an object’s ACL evaluating ACEs in order until all requested rights have been granted, a requested right has been explicitly denied by an ACE or the end of the list has been reached. In case there is no ACL or the permissions granted by the ACL are not sufficient to fulfill the request, macOS next evaluates the object’s UNIX permissions. Therefore ACLs always have precedence over UNIX permissions.

ZFS ACLs

ZFS ACLs closely match macOS ACLs. Both offer mostly identical fine grained permissions and inheritance settings.

POSIX ACLs

Overview

Compared to macOS or NFSv4 ACLs, POSIX ACLs represent a different, less versatile approach to overcome the limitations of the traditional UNIX permissions. Implementations are based on the withdrawn POSIX 1003.1e standard.

The standard defines two types of ACLs. Files and directories can have access ACLs which are consulted for access checks. Directories can also have default ACLs irrelevant to access checks. When a new object is created inside a directory with a default ACL, the default ACL is applied to the new object as its access ACL. Subdirectories inherit default ACLs from their parent. There are no further mechanisms of inheritance control.

Architectural differences between POSIX ACLs and macOS ACLs especially involve:

POSIX 1003.1e defines 6 different types of ACL entries. The first three types are used to integrate standard UNIX permissions. They form a minimal ACL, their presence is mandatory and only one entry of each type is allowed within an ACL.

The remaining entry types expand the traditional permissions model:

In order to maintain compatibility with applications not aware of ACLs, POSIX 1003.1e changes the semantics of system calls and utilities which retrieve or manipulate an object’s UNIX permissions. In case an object only has a minimal ACL, the group permissions bits of the UNIX permissions correspond to the value of the ACL_GROUP_OBJ entry.

However, if the ACL also contains an ACL_MASK entry, the behavior of those system calls and utilities is different. The group permissions bits of the UNIX permissions correspond to the value of the ACL_MASK entry, i. e. calling “chmod g-w” will not only revoke write access for the group, but for all entities which have been granted write access by ACL_USER or ACL_GROUP entries.

Mapping POSIX ACLs to macOS ACLs

When a client wants to read an object’s ACL, afpd maps its POSIX ACL onto an equivalent macOS ACL. Writing an object’s ACL requires afpd to map an macOS ACL onto a POSIX ACL. Due to architectural restrictions of POSIX ACLs, it is usually impossible to find an exact mapping so that the result of the mapping process will be an approximation of the original ACL’s semantic.

As macOS clients aren’t aware of the POSIX 1003.1e specific relationship between UNIX permissions and ACL_MASK, afpd does not expose this feature to the client to avoid compatibility issues and handles *unix permissions and ACLs the same way as Apple’s reference implementation of AFP does. When an object’s UNIX permissions are requested, afpd calculates proper group rights and returns the result together with the owner’s and everybody’s access rights to the caller via “permissions” and “ua_permissions” members of the FPUnixPrivs structure (see Apple Filing Protocol Reference, page 181). Changing an object’s permissions, afpd always updates ACL_USER_OBJ, ACL_GROUP_OBJ and ACL_OTHERS. If an ACL_MASK entry is present too, afpd recalculates its value so that the new group rights become effective and existing entries of type ACL_USER or ACL_GROUP stay intact.

Directory Cache Tuning

The dircache (Directory Cache) in Netatalk provides a cache of the full paths for all files and directories enumerated by a user. Once populated during the first directory enumeration, subsequent enumerations can be delivered directly from memory. This enables fast responses even when working with very large directories. When a user renames/deletes files and directories, the dircache entries are updated/removed immediately.

While the dircache is kept in sync with AFP operations, it is unable to detect external changes outside of Netatalk (eg, local file changes, or changes using other file sharing services like Samba). As a result, Netatalk performs a stat() operation to validate a dircache entry is still valid, every single time a cache entry is read. This impacts the benefits and effectiveness of the dircache.

For setups where Netatalk is the only accessor, or where external file changes are infrequent, the additional file stat operations are mostly redundant, and can be safely tuned down. This significantly reduces IO operations on storage/page-cache/ZFS ARC layers.

If a dircache entry is found to be stale when used, the invalid_on_use counter is incremented, and the dircache entry is transparently rebuilt. You can use the invalid_on_use counter to tune dircache validation freq.

dircachesize = number (G)

Maximum possible entries in the directory cache. The cache stores directories and files. It is used to cache the full path to directories and CNIDs which considerably speeds up directory enumeration. Given value is rounded up to nearest power of 2. Each entry takes about 100 bytes, which is not much, but remember that every afpd child process for every connected user has its own cache.

Default: 8192, Maximum size: 131072.

dircache validation freq = number (G)

Directory cache validation frequency for external change detection. Controls how often cached entries are validated against the filesystem to detect changes made outside of netatalk (e.g., direct filesystem modifications by other processes). A value of 1 means validate cache every access (default for backward compatibility which performs a stat() operation on the storage object to validate the cache every cache read), higher values validate less frequently. For example, 5 means validate cache entries every 5th access. Higher values improve performance and significantly reduce storage IO and page cache stress, but may delay detection of changes external from Netatalk.

Default: 1, Range: 1-100.

Internal netatalk operations (file/directory create, delete, rename) always invalidate dircache entries immediately regardless of this setting. If Netatalk is the only process accessing the volume you can safely set a value of 100 for maximum performance.

dircache metadata window = number (G)

Time window in seconds for distinguishing metadata-only changes from content changes in directories. When a directory’s ctime changes within this window from the current time, the change is considered potentially metadata-only (permissions, extended attributes) rather than content modification. Smaller values are more conservative but may cause unnecessary cache invalidation. Larger values improve performance but may retain stale entries longer.

Default: 300 seconds (5 minutes). Range: 60-3600 seconds.

If Netatalk is the only process accessing the volume you can safely set a value of 3600.

dircache metadata threshold = number (G)

Maximum time difference in seconds between cached and current directory ctime to be considered a metadata-only change. Works together with dircache metadata window to avoid invalidating cache entries for minor metadata modifications. This prevents cache invalidation for small, recent ctime changes that are likely due to more frequent permission or extended attribute modifications rather than directory content changes.

Default: 60 seconds (1 minute). Range: 10-1800 seconds.

If Netatalk is the only process accessing the volume you can safely set a value of 1800.

Example (Netatalk only access to volume): dircache validation freq = 100 dircache metadata window = 3600 dircache metadata threshold = 1800

Note: Monitor dircache effectiveness by checking Netatalk log files for “dircache statistics:” lines when afpd shuts down gracefully (user disconnects).

Filesystem Change Events

Netatalk includes a nifty filesystem change event (FCE) mechanism where afpd processes notify interested listeners about certain filesystem events by UDP network datagrams.

This feature is disabled by default. To enable it, set the fce listener option in afp.conf to the hostname or IP address of the host that should listen for FCE events.

Netatalk distributes a simple FCE listener application called fce_listen. It will print out any UDP datagrams received from the AFP server. Its source code also serves as documentation for the format of the UDP packets.

FCE v1

The currently supported FCE v1 events are:

FCE v2

FCE v2 events provide additional context such as the user who performed the action. When using FCE v2 you also get the following events:

Spotlight Compatible Search

You can enable Netatalk’s Spotlight compatible search and indexing either globally or on a per volume basis with the spotlight option.

WARNING: Once Spotlight is enabled for a single volume, all other volumes for which spotlight is disabled won’t be searchable at all.

The dbus-daemon binary has to be installed for Spotlight feature. The path to dbus-daemon is determined at compile time the dbus-daemon build system option.

In case the dbus-daemon binary is installed at the other path, you must use the global option dbus daemon to point to the path, e.g. for Solaris with Tracker from OpenCSW:

dbus daemon = /opt/csw/bin/dbus-daemon