NFSv4 mount incorrectly shows all files with ownership as nobody:nobody

Solution Verified - Updated 6 Jun 2025

Environment

Red Hat Enterprise Linux 6 and later
NFSv4 share being exported from an NFSv4 capable NFS server

Issue

From the client, the mounted NFSv4 share has ownership for all files and directories listed as nobody:nobody instead of the actual user that owns them on the NFSv4 server, or who created the new file and directory.
Seeing nobody:nobody permissions on nfsv4 shares on the NFS client. Also seeing the following error in /var/log/messages:

How to configure ID mapping for NFSv4

nss_getpwnam: name 'root@example.com' does not map into domain 'localdomain'

Resolution

Modify the /etc/idmapd.conf with the proper domain (FQDN), on both the client and server. In this example, the proper domain is "example.com" so the "Domain =" directive within /etc/idmapd.conf should be modified to read:
```
Domain = example.com
```
Note:
- If using a Net App Filer, the NFS.V4.ID.DOMAIN parameter must be set to match the "Domain =" parameter on the client.
- If using a Solaris machine as the NFS server, the NFSMAPID_DOMAIN value in /etc/default/nfs must match the RHEL clients Domain.

RHEL 7 and newer

Beginning with RHEL 7, the service is named nfs-idmapd.service, and is restarted with the following command:
```
# systemctl restart nfs-idmapd.service
```
Ensure the client and server have matching UIDs and GIDs. It is a common misconception that the UIDs and GIDs can differ when using NFSv4. The sole purpose of ID mapping is to map an ID to a name and vice versa. ID mapping is not intended as some sort of replacement for managing ID's.
With latest kernel on RHEL 7, only the NFSv4 server uses rpc.idmapd. The NFSv4 client uses the keyring-based idmapper nfsidmap. nfsidmap is a stand-alone program that is called by the kernel on-demand to perform ID mapping; it is not a daemon.
On Red Hat Enterprise Linux 6.3 and higher, if the above settings have been applied and UIDs/GIDs are matched on server and client and users are still being mapped to nobody:nobody then a clearing of the idmapd cache may be required:
```
# nfsidmap -c 
```
Note: The above command is only necessary on systems that use the keyring-based ID mapper, i.e. NFS CLIENTS running RHEL 6.3 and higher. On RHEL 6.2 and older NFS CLIENTS as well as all NFS SERVERS running RHEL, the cache should be cleared out when rpc.idmapd is restarted.

Another check, see if the passwd:, shadow: and group: settings are set correctly in the /etc/nsswitch.conf file on both Server and Client.

RHEL versions up through RHEL 6

On Red Hat Enterprise Linux 6.2 and older, to put the changes into effect restart the rpcidmapd service and remount the NFSv4 filesystem:
```
# service rpcidmapd restart
# mount -o remount /nfs/mnt/point
```
Note: It is only necessary to restart rpc.idmapd service on systems where rpc.idmapd is actually performing the ID mapping. On RHEL 6.3 and newer NFS CLIENTS, the maps are stored in the kernel keyring and the ID mapping itself is performed by the /sbin/nfsidmap program. On older NFS CLIENTS (RHEL 6.2 and older) as well as on all NFS SERVERS running RHEL, the ID mapping is performed by rpc.idmapd.

Disabling ID mapping

Note: In order to properly disable ID mapping, it must be disabled on both the NFS client and NFS server.

By default, RHEL6.3 and newer NFS clients and servers disable idmapping when utilizing the AUTH_SYS/UNIX authentication flavor by enabling the following booleans:

  NFS client 
  # echo 'Y' > /sys/module/nfs/parameters/nfs4_disable_idmapping 

  NFS server
  # echo 'Y' > /sys/module/nfsd/parameters/nfs4_disable_idmapping

If using a Net App filer, the options nfs.v4.id.allow_numerics on command can be used to disable idmapping. More information can be found Content from library.netapp.com is not included.here.
With this boolean enabled, NFS clients will instead send numeric UID/GID numbers in outgoing attribute calls and NFS servers will send numeric UID/GID numbers in outgoing attribute replies.

If NFS clients sending numeric UID/GID values in a SETATTR call receive an NFS4ERR_BADOWNER reply from the NFS server, clients will re-enable ID mapping and send user@domain strings for that specific mount from that point forward.
We can make the option nfs4_disable_idmapping persistent across reboot.
After the above value has been changed, for the setting to take effect for any NFS server export mounted on the NFS client, you must unmount all NFS mount points for the given NFS server, and then re-mount them. If you have auto mounts, stop all processes accessing the mounts and allow the automount daemon to unmount them. Once all NFS mount points are gone to the desired NFS server, remount the NFS mount points and the new setting should be in place. If this is too problematic, you may want to schedule a reboot of the NFS client.
To verify the setting has been changed properly, you can look inside the /proc/self/mountstats file 'caps' line, which contains a hex value of 2 bytes (16 bits). This is the line that shows the NFS server's "capabilities", and the most significant bit #15 is the one which represents whether idmapping is disabled or not (the NFS_CAP_UIDGID_NOMAP bit - see the Root Cause section)
```
# cat /sys/module/nfs/parameters/nfs4_disable_idmapping
Y
# umount /mnt
# mount rhel6u6-node2:/exports/nfs4 /mnt
# grep -A 5 rhel6u6-node2 /proc/self/mountstats | egrep '(rhel6u6-node2|caps:)'
  device rhel6u6-node2:/exports/nfs4 mounted on /mnt with fstype nfs4 statvers=1.0
  caps:   caps=0xffff,wtmult=512,dtsize=32768,bsize=0,namlen=255
                 ^
```
Example of nfs4_disable_idmapping = 'N'
```
# echo N > /sys/module/nfs/parameters/nfs4_disable_idmapping
#  umount /mnt
#  mount rhel6u6-node2:/exports/nfs4 /mnt
#  grep -A 5 rhel6u6-node2 /proc/self/mountstats | egrep '(rhel6u6-node2|caps:)'
 device rhel6u6-node2:/exports/nfs4 mounted on /mnt with fstype nfs4 statvers=1.0
  caps:   caps=0x7fff,wtmult=512,dtsize=32768,bsize=0,namlen=255
                 ^
```
Note: To force ONLY numeric IDs to be used on the client, add RPCIDMAPDARGS="-C" to the etc/sysconfig/nfs file and restart the rpcidmapd service (nfs-idmapd.service beginning with RHEL 7). See man rpc.idmapd for more information.

Note: This option can only be used with AUTH_SYS/UNIX authentication flavors, if you wish to use something like Kerberos, ID mapping must be used.

Root Cause

NFSv4 utilizes ID mapping to ensure permissions are set properly on exported shares, if the domains of the client and server do not match then the permissions are mapped to nobody:nobody.

NFS_CAP_UIDGID_NOMAP bit

The nfs4_disable_idmapping is a module parameter which is read only one time, at the point at which the kernel sets up the data structure that represents an NFS server. Once it is read, a flag is set in the nfs_server structure NFS_CAP_UIDGID_NOMAP.

  #define NFS_CAP_UIDGID_NOMAP    (1U << 15)

  static int nfs4_init_server(struct nfs_server *server,
                const struct nfs_parsed_mount_data *data)
  {
        struct rpc_timeout timeparms;
        int error;

        dprintk("--> nfs4_init_server()\n");

        nfs_init_timeout_values(&timeparms, data->nfs_server.protocol,
                        data->timeo, data->retrans);

        /* Initialise the client representation from the mount data */
        server->flags = data->flags;
        server->caps |= NFS_CAP_ATOMIC_OPEN|NFS_CAP_CHANGE_ATTR|NFS_CAP_POSIX_LOCK;
        if (!(data->flags & NFS_MOUNT_NORDIRPLUS))
                        server->caps |= NFS_CAP_READDIRPLUS;
        server->options = data->options;

        /* Get a client record */
        error = nfs4_set_client(server,
                        data->nfs_server.hostname,
                        (const struct sockaddr *)&data->nfs_server.address,
                        data->nfs_server.addrlen,
                        data->client_address,
                        data->auth_flavors[0],
                        data->nfs_server.protocol,
                        &timeparms,
                        data->minorversion);
        if (error < 0)
                goto error;

        /*
         * Don't use NFS uid/gid mapping if we're using AUTH_SYS or lower
         * authentication.
         */
        if (nfs4_disable_idmapping && data->auth_flavors[0] == RPC_AUTH_UNIX)  <--- set a flag based on the module parameter
                server->caps |= NFS_CAP_UIDGID_NOMAP;   <-------------------------- flag set

        if (data->rsize)
                server->rsize = nfs_block_size(data->rsize, NULL);
        if (data->wsize)
                server->wsize = nfs_block_size(data->wsize, NULL);

        server->acregmin = data->acregmin * HZ;
        server->acregmax = data->acregmax * HZ;
        server->acdirmin = data->acdirmin * HZ;
        server->acdirmax = data->acdirmax * HZ;

        server->port = data->nfs_server.port;

        error = nfs_init_server_rpcclient(server, &timeparms, data->auth_flavors[0]);

  error:
        /* Done */
        dprintk("<-- nfs4_init_server() = %d\n", error);
        return error;
  }

This flag is later checked when deciding whether to use numeric uid or gids or to use idmapping.

  int nfs_map_uid_to_name(const struct nfs_server *server, __u32 uid, char *buf, size_t buflen)
  {
        struct idmap *idmap = server->nfs_client->cl_idmap;
        int ret = -EINVAL;

        if (!(server->caps & NFS_CAP_UIDGID_NOMAP)) {  <------------ CHECK FLAG, DECIDE whether to call idmapper
                ret = nfs_idmap_lookup_name(uid, "user", buf, buflen);
                if (ret < 0)
                        ret = nfs_idmap_name(idmap, &idmap->idmap_user_hash, uid, buf);
        }
        if (ret < 0)
                ret = nfs_map_numeric_to_string(uid, buf, buflen);
        return ret;
  }
  int nfs_map_gid_to_group(const struct nfs_server *server, __u32 gid, char *buf, size_t buflen)
  {
        struct idmap *idmap = server->nfs_client->cl_idmap;
        int ret = -EINVAL;

        if (!(server->caps & NFS_CAP_UIDGID_NOMAP)) {  <------------ CHECK FLAG, DECIDE whether to call idmapper
                ret = nfs_idmap_lookup_name(gid, "group", buf, buflen);
                if (ret < 0)
                        ret = nfs_idmap_name(idmap, &idmap->idmap_group_hash, gid, buf);
        }
        if (ret < 0)
                ret = nfs_map_numeric_to_string(gid, buf, buflen);
        return ret;
  }
  "fs/nfs/idmap.c" 872L, 21804C

For more information on NFSv4 ID mapping in Red Hat Enterprise Linux, see <https://access.redhat.com/articles/2252881>

Diagnostic Steps

Debugging/verbosity can be enabled by editing /etc/sysconfig/nfs:
```
RPCIDMAPDARGS="-vvv"
```

The following output is shown in /var/log/messages when the mount has been completed and the system shows nobody:nobody as user and group permissions on directories and files:

Jun  3 20:22:08 node1 rpc.idmapd[1874]: nss_getpwnam: name 'root@example.com' does not map into domain 'localdomain' 
Jun  3 20:25:44 node1 rpc.idmapd[1874]: nss_getpwnam: name 'root@example.com' does not map into domain 'localdomain'

Collect a tcpdump of the mount attempt:

# tcpdump -s0 -i {INTERFACE} host {NFS.SERVER.IP} -w /tmp/{casenumber}-$(hostname)-$(date +"%Y-%m-%d-%H-%M-%S").pcap &

If a TCP packet capture has been obtained, check for a nfs.nfsstat4 packet that has returned a non-zero response equivalent to 10039 (NFSV4ERR_BADOWNER).

From the NFSv4 RFC:

  NFS4ERR_BADOWNER        = 10039,/* owner translation bad   */

  NFS4ERR_BADOWNER      An owner, owner_group, or ACL attribute value
                        can not be translated to local representation.

SBR

Filesystem

Product(s)

Red Hat Enterprise Linux

Components

nfs-utils

Category

Troubleshoot

Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.