Rollback Root Filesystem using Btrfs

Do all of this as root

sudo su -

Find the label associated with your btrfs volume

# blkid | grep btrfs
/dev/sda3: LABEL="fedora" UUID="31347ba9-6ca8-41e1-983b-716e4e997a24" UUID_SUB="47b6f9f4-b9c5-4aac-9637-a274d20b2bea" TYPE="btrfs" PARTUUID="c54db764-03"

Mount the btrfs volume (this in not the same as your root filesystem which is a subvolume of the btrfs root)

mkdir /btrfs
mount -t btrfs -o subvolid=0 LABEL="fedora" /btrfs
cd /btrfs

Optionally, add this mount to /etc/fstab

echo "LABEL=\"fedora\" /btrfs btrfs defaults,noauto,subvolid=0 0 0" >> /etc/fstab

Find which subvolume is your root filesystem

# btrfs subvolume show /.
 Name: root
 uuid: 28ce82e4-70d9-2f4c-aefb-c2ed1f1066c2
 Parent uuid: ed50ebba-4e51-b142-a2f5-a6f1782d8350
 Creation time: 2014-12-29 23:45:52
 Object ID: 272
 Generation (Gen): 452
 Gen at creation: 350
 Parent: 257
 Top Level: 257
 Flags: -

Create a snapshot of that subvolume

btrfs subvolume snapshot root root-$(date +%Y%m%d)

To rollback the root filesystem to the snapshot, set the subvolume id the snapshot as the default for the subvolume

# btrfs subvolume list /btrfs
ID 272 gen 425 top level 5 path root
ID 280 gen 350 top level 257 path root-20141230

# btrfs subvolume set-default 280 /.

Remove all the “rootflags=subvol=root” arguments from /boot/grub2/grub.cfg. If you don’t do this, it will disregard the default subvolume id we just set and always boot into the root subvolume.

sed -i 's/rootflags=subvol=root //' /boot/grub2/grub.cfg

NOTE: This change is temporary, as anything that regenerates the grub config file will undo these changes.  However, once rebooted into the snapshot, any regeneration of the grub config will set the subvol parameter to the name of the snapshot.

Reboot.  Once booted, ensure that the snapshot is, in fact, the root filesystem

# btrfs subvolume show /.
 Name: root-20141230
 uuid: ed50ebba-4e51-b142-a2f5-a6f1782d8350
 Parent uuid: b39f90ad-2bbc-b542-b0ac-d6204a45fc42
 Creation time: 2014-12-29 15:45:41
 Object ID: 280
 Generation (Gen): 350
 Gen at creation: 156
 Parent: 257
 Top Level: 257
 Flags: readonly

If you do not wish to retain the old root subvolume, it can be removed

cd /btrfs
btrfs subvolume delete root

The system is now running on (and modifying) the snapshot subvolume.  If you wish to maintain a rollback snapshot, you need to take another snapshot.

Build Upstream Linux Kernel for Fedora 21

Install all the packages required for building the kernel

sudo yum-builddep kernel

Download the upstream kernel source for the kernel version you are currently running from

wget$(uname -r | cut -f1 -d'-').tar.xz

Extract the source and enter the directory

tar xf linux-*
cd linux-*

Copy the distribution kernel config to the source directory as .config

cp /boot/config-$(uname -r) .config

Make sure the config file works against the source (select the default when it asks for the number of CPUs unless you have specific needs)

make oldconfig

Open the kernel config menu

make menuconfig

Go to Kernel Hacking -> Compile-time checks and compiler options and unmark Compile the kernel with debug info.  The debug info is not needed and adds significant size to the compiled objects.

Set MAKEFLAGS so it builds in parallel using all the cores on the system


Build the kernel and modules (this will take a while)

make && make modules

Become root

sudo bash

Install the modules

make modules_install

Copy the kernel image to /boot

cp arch/x86/boot/bzImage /boot/vmlinuz-$(make kernelrelease)

Create the initramfs

dracut /boot/initramfs-$(make kernelrelease).img $(make kernelrelease)

Update the boot configuration

grub2-mkconfig > /boot/grub2/grub.cfg


When the grub menu appears, select your new upstream kernel (the one without fc21 in the kernel version)

Set Up a Virtual Mail Server on Fedora 21 using Dovecot, Postfix, OpenDKIM, and SpamAssassin

These are instructions on how to set up virtual mail server in Fedora 21.

In this case, “virtual” is used to indicate that mail being handled by the server is not addressed to its hostname and/or domain and the users are not local to the server.

This configuration is for a incoming mail server, not an outgoing mail relay.

I find that is it more straightforward from a policy standpoint to have your incoming mail server and your outgoing mail relay to be two different systems.  In my case, I have a as an incoming mail server and an as an outgoing mail relay.

All the commands are run as root.

sudo su -

Install dovecot and postfix

yum install postfix dovecot -y
systemctl enable dovecot postfix

Postifx (MTA)

Change the system Mail Transfer Agent (MTA) to postfix

# alternatives --config mta

There are 2 programs which provide 'mta'.

 Selection Command
*+ 1 /usr/sbin/sendmail.ssmtp
 2 /usr/sbin/sendmail.postfix

Enter to keep the current selection[+], or type selection number: 2

Convert the /etc/aliases database to the new MTA format


Configure postfix add adding/modifying the following options to /etc/postfix/, replacing with your domain and replacing with your network(s):

inet_interfaces = all
mydestination = localhost
mynetworks =,

virtual_transport = lmtp:unix:private/dovecot-lmtp
virtual_mailbox_domains =
virtual_mailbox_maps = hash:/etc/postfix/mailboxes

smtpd_banner = $myhostname ESMTP
smtpd_helo_required = yes
smtpd_helo_restrictions =
smtpd_sender_restrictions =
smtpd_recipient_restrictions =

smtpd_tls_ciphers = high

Create the /etc/postfix/mailboxes file with the following contents (add entries for your accounts).  The second value “OK” can be anything an has no purpose other than being a placeholder.  The virtual_mailbox_maps serves only allows postfix to determine if an account for this mail server or not. OK

Create the mailboxes database

postmap /etc/postfix/mailboxes

Optional (virtual aliases)

You can also create a virtual_alias_maps by adding this to /etc/postfix/

virtual_alias_maps = hash:/etc/postfix/virtual

Creating a /etc/postfix/virtual file with the alias mapping

And creating the virtual database

 postmap /etc/postfix/virtual


Dovecot has several of responsibilities

  • Acts as a Mail Delivery Agent, accepting mail from postfix
  • Manages filtering and storage of mail
  • Provides user authentication
  • Serves POP/IMAP services to users

Set the protocols you want to support in /etc/dovecot/dovecot.conf

protocols = imap lmtp

Set the mail_location in  /etc/dovecot/conf.d/10-mail.conf.  Using our user, the mail directory would be /var/mail/vhosts/

mail_location = maildir:/var/mail/vhosts/%d/%n

Modify the service lmtp stanza in /etc/dovecot/conf.d/10-master.conf.  LMTP is Local Mail Transport Protocol and is an simple method for postfix to relay the mail to Dovecot.  Dovecot receives mail from postfix on this socket.

unix_listener /var/spool/postfix/private/dovecot-lmtp {
  mode = 0666
  user = postfix
  group = postfix

In /etc/dovecot/conf.d/10-auth.conf

Set disable_plaintext_auth

 disable_plaintext_auth = yes

Comment out

!include auth-system.conf.ext


 #!include auth-passwdfile.conf.ext

Replace the content of /etc/dovecot/conf.d/auth-passwdfile.conf.ext with the following

passdb {
  driver = passwd-file
  args = username_format=%u /etc/dovecot/users

userdb {
  driver = static
  args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n

Create an encrypted password for

# doveadm pw -s SHA256-CRYPT
Enter new password: 
Retype new password: 

Create  /etc/dovecot/users and replace password string with output from doveadm pw{SHA256-CRYPT}$5$0NAlFLmqYAmIxuUu$XKJ17uS5oN9iW9lIVNTi1Zsp2jlh6tQ.P2pbY1bJro4

Create vmail user for delivering mail

useradd -d /var/mail/vhosts vmail
mkdir -p /var/mail/vhosts
chown vmail:vmail /var/mail/vhosts


DomainKeys Identified Mail (DKIM) is a message signing system that can allows mail servers to verify that messages are actually from the domain that claimed to send them.  The signing helps in detecting forged email and spamassassin uses it in spam detection.

Install opendkim

yum install opendkim -y

By default, opendkim runs in verify mode with no signing, which is what we want.

Set milter (mail filter) for postfix in /etc/postfix/

smtpd_milters = inet:localhost:8891
milter_default_action = accept

Enable the service

systemctl enable opendkim


Install spamassassin and postfix milter

yum install spamassassin spamass-milter-postfix -y

Update the spamassassin database


Set the spamassassin milter for postfix in /etc/postfix/ (after the opendkim milter we already set)

smtpd_milters = inet:localhost:8891, unix:/run/spamass-milter/postfix/sock

Enable services

systemctl enable spamassassin spamass-milter

Firewall (optional)

Probably a good idea to setup a basic iptables firewall.  This lets SSH, SMTP, IMAP, and IMAPS through.

yum remove firewalld -y
yum install iptables-services -y
systemctl enable iptables

iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
iptables -A INPUT -p tcp -m tcp --dport 25 -j ACCEPT
iptables -A INPUT -p tcp -m tcp --dport 143 -j ACCEPT
iptables -A INPUT -p tcp -m tcp --dport 993 -j ACCEPT
iptables -A INPUT -j DROP

iptables-save > /etc/sysconfig/iptables

Start Everything Up

Start all services

systemctl start opendkim spamassassin spamass-milter dovecot postfix

Don’t forget to configure the DNS server for to include both an A record for the mail server name and an MX record routing to your mail server name.

Using systemd-nspwan for lightweight container in Fedora 21

Every once and a while, I find the need for a temporary Fedora environment to do a certain thing, like build a project from source that has a ton of dependencies that I don’t want to install on my real Fedora workstation.  systemd has a great tool for setting up such an environment called systemd-nspawn.   It sets up a glorified chroot environment with all the benefits of containers, namely a separate separate process and IPC namespace so you can run an init system.  Best of all, it is easy to setup and use.  All you need is yum and systemd, which, if you are on Fedora, are definitely installed.

This needs be done as root

sudo su -

First build your root filesystem

# yum -y --releasever=21 --nogpg --installroot=/srv/f21 --disablerepo='*' --enablerepo=fedora --enablerepo=updates install systemd passwd yum fedora-release @standard

Because the host selinux policy only has entries for the host environment, you will need to create an selinux policy for all files in the root directory for the container.  This policy basically labels all files with the same label, effectively disabling selinux for the container.

# semanage fcontext -a -t svirt_sandbox_file_t "/srv/f21(/.*)?"
# restorecon -R /srv/f21/

You can verify that the new label has been applied

# ls -lZ /srv/
dr-xr-xr-x. root root system_u:object_r:svirt_sandbox_file_t:s0 f21

Set the root password for the container

# systemd-nspawn -D /srv/f21
Spawning container f21 on /srv/f21.
Press ^] three times within 1s to kill container.
-bash-4.3# passwd
Changing password for user root.
New password: 
Retype new password: 
passwd: all authentication tokens updated successfully.
-bash-4.3# exit

Container f21 exited successfully.

Boot the container

# systemd-nspawn -bD /srv/f21
Spawning container f21 on /srv/f21.
Press ^] three times within 1s to kill container.
Detected virtualization 'systemd-nspawn'.
Detected architecture 'x86-64'.

Welcome to Fedora 21 (Twenty One)!

Initializing machine ID from random generator.
Cannot add dependency job for unit display-manager.service, ignoring: Unit display-manager.service failed to load: No such file or directory.
[ OK ] Reached target Remote File Systems.
[ OK ] Reached target Encrypted Volumes.
[ OK ] Reached target Swap.
[ OK ] Created slice Root Slice.
[ OK ] Started Login Service.
[ OK ] Started Cleanup of Temporary Directories.
[ OK ] Started System Logging Service.

Fedora release 21 (Twenty One)
Kernel 3.17.4-302.fc21.x86_64 on an x86_64 (console)

f21 login: root

When you are done, shut down the container

-bash-4.3# poweroff
[ OK ] Started Show Plymouth Power Off Screen.
 Stopping Create Volatile Files and Directories...
[ OK ] Stopped Create Volatile Files and Directories.
 Stopping Create System Users...
[ OK ] Stopped Create System Users.
[ OK ] Stopped target Local File Systems.
 Unmounting /run/user/0...
 Unmounting /proc/sys/kernel/random/boot_id...
 Unmounting Temporary Directory...
 Stopping Configure read-only root support...
[ OK ] Stopped Configure read-only root support.
[ OK ] Reached target Shutdown.
Sending SIGTERM to remaining processes...
Sending SIGKILL to remaining processes...
Powering off.

Container f21 has been shut down.

Fix “Authenticating with Gearbox Shift” in Borderlands in Fedora

This problem is due to the way the different distributions handle SSL root CAs for the system.  Borderlands: The Pre-Sequel (BL:TPS) looks for the root certificate in a specific file.   This file exists in Ubuntu, the target for Steam and, thus, the port.  In Fedora, a symbolic link to the root certificate file is enough to fix this

sudo ln -s /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem /etc/pki/tls/certs/157753a5.0

EDIT For F22 it seems that this is the correct symlink to the cert (thanks Eric)

sudo ln -s /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem /etc/pki/tls/certs/2c543cd1.0

Set the launch options

__GL_THREADED_OPTIMIZATIONS=1 SSL_CERT_DIR="/etc/pki/tls/certs" %command%


Remove unneeded packages from GNOME

This is solely my preference, but many of the apps shipped with GNOME are limited and bordering on useless compared to other more full featured applications (boxes vs virt-manager, totem/rhythmbox vs gnome-mplayer, gedit vs gvim)

Remove the unneeded

sudo yum autoremove empathy.x86_64 evolution gnome-boxes.x86_64 devassistant.noarch gnome-documents.x86_64 gnome-weather.noarch shotwell.x86_64 gnome-contacts.x86_64 rhythmbox.x86_64 cheese totem -y

Install the better alternatives

sudo yum install @Virtualization virt-manager gnome-mplayer vim-X11 -y

Some that I didn’t need by you might

sudo yum autoremove orca.noarch transmission

Reset your default applications for file types by right clicking on a file the specified type, and selecting Properties. Under the Open With tab, select the appropriate application to handle the file type.


Replace Nautilus (Files) with Nemo in GNOME

Install Nemo

sudo yum install nemo nemo-fileroller nemo-preview -y

Set Nemo as the default handler for inode/directory types

xdg-mime default nemo.desktop inode/directory

Test (should open your home directory with Nemo)

xdg-open $HOME

Turn off desktop icon handling by Files (Nautilus)

gsettings set org.gnome.desktop.background show-desktop-icons false

Turn on desktop icon handling by Nemo

gsettings set org.nemo.desktop show-desktop-icons true

Start Nemo by pressing Alt-F2 and running nemo -n

If the icon text is not readable against your background, you can adjust the style in the ~/.config/gtk-3.0/gtk.css file. This example will make the text color white.

cat << EOF > ~/.config/gtk-3.0/gtk.css
.nemo-desktop.nemo-canvas-item {
  color: #FFFFFF;
  text-shadow: 1px 1px @desktop_item_text_shadow;

In order for nemo to show your desktop icons for each new login, you need to autostart it by adding a nemo.desktop file to your ~/.config/autostart directory.

cat << EOF > ~/.config/autostart/nemo.desktop
[Desktop Entry]
Exec=nemo -n

Optionally, remove nautilus

sudo yum autoremove nautilus -y

You can control the icon size by opening Nemo, going to Edit->Preferences, under the Views tab with the Default zoom level for Icon View Defaults.


Install the NVidia Driver in Fedora

From NVidia

Download the latest NVidia driver from

Install packages required for building

sudo yum install binutils gcc make kernel-devel kernel-headers -y

Logout and switch to a vterm using Ctrl-Alt-F2

Login and switch to multi-user mode (the installer can’t run when the X server is running)

sudo systemctl isolate

Change to the directory that contains the NVidia driver file

Make the file executable and run it as root

chmod +x NVIDIA-Linux*
sudo ./NVIDIA-Linux*

Accept all the default options during install

Switch back to graphical mode

sudo systemctl isolate

If you update your kernel you will need to rebuild the NVidia driver as well so keep this installer around.

From rpm-fusion

Download the non-free rpm-fusion repo RPM appropriate for your version from

Install the RPM

sudo yum localinstall rpmfusion-nonfree-release-* -y

Install the NVidia driver

sudo yum install kmod-nvidia -y

There is a kmod-nvidia package per kernel release and they must match. If you do a yum update, there is a chance that there is a new kernel package but not a new kmod-nvidia package to match it (yet). If that is the case, your machine will fail to enter graphical mode. Make sure that there is a kmod-nvidia package that matches your kernel in the non-free repo before upgrading your kernel. If you do this accidentally, simply reboot into the previously working kernel and remove the updated kernel until such time as there is a kmod-nvidia package for it.