Posted: November 16, 2020 | | by Tyler Carrigan (Editorial Team, Red Hat)
Image
Security Enhanced Linux (SELinux)is a security framework that allows and denies access to applications, files, etc. within a Linux system. For a full SELinux overview, see What is SELinux.
For this article, we will look at what happens when access to a desired file or application is denied. What messages will you see, where are these messages stored, and what does the information actually mean? What are some of the abnormal situations, caused by false positives and ghost denials, that you might find yourself in? We will explore all of that here.
What is a denial?
A denial is the event generated anytime that a service, application, file, etc. is denied access by the SELinux system. When this happens, the denial is cached in the Access Vector Cache (AVC). You will sometimes see a denial message referred to as an AVC denial.
Where are denials logged?
Now, these AVC denials, much like everything else in Linux, are logged by the system. Where those messages are logged varies depending on which system daemons are running.
- auditd on - /var/log/audit/audit.log
- auditd off; rsyslogd on - /var/log/messages
- setroubleshootd, rsyslogd, and auditd on - Both locations, though the messages in /var/log/messages are easier to make sense of
What will a denial look like?
You will sometimes see a denial warning on your desktop. When you select show, this alert will give you details as to what went wrong and even sometimes a solution to solve the problem.
Another thing you can do is run the following commands to pull out the denial messages:
[root@server ~]# grep "SELinux is preventing" /var/log/messagesHere the output is in more approachable language, and if you read carefully, a solution is presented:
Sep 22 13:35:24 server setroubleshoot[3999]: SELinux is preventing rhsmcertd-worke from read access on the file virt.module. For complete SELinux messages run: sealert -l 97a1c0df-81ed-4c08-ba27-41c5067b713bSep 22 13:35:24 server platform-python[3999]: SELinux is preventing rhsmcertd-worke from read access on the file virt.module.#012#012***** Plugin catchall_boolean (89.3 confidence) suggests ******************#012#012If you want to allow daemons to dump core#012Then you must tell SELinux about this by enabling the 'daemons_dump_core' boolean.#012#012Do#012setsebool -P daemons_dump_core 1#012#012***** Plugin catchall (11.6 confidence) suggests **************************#012#012If you believe that rhsmcertd-worke should be allowed read access on the virt.module file by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'rhsmcertd-worke' --raw | audit2allow -M my-rhsmcertdworke#012# semodule -X 300 -i my-rhsmcertdworke.pp#012This output is actually really helpful, as the first entry above gives us the following:
For complete SELinux messages run: sealert -l 97a1c0df-81ed-4c08-ba27-41c5067b713bIf you run the suggested command, you are given a very verbose yet straightforward summary of what happened and what you can do to remedy the issue. Seen below:
[root@server ~]# sealert -l 97a1c0df-81ed-4c08-ba27-41c5067b713b SELinux is preventing rhsmcertd-worke from read access on the file virt.module. ***** Plugin catchall_boolean (89.3 confidence) suggests ****************** If you want to allow daemons to dump core Then you must tell SELinux about this by enabling the 'daemons_dump_core' boolean. Do setsebool -P daemons_dump_core 1 ***** Plugin catchall (11.6 confidence) suggests ************************** If you believe that rhsmcertd-worke should be allowed read access on the virt.module file by default. Then you should report this as a bug. You can generate a local policy module to allow this access. Do allow this access for now by executing: # ausearch -c 'rhsmcertd-worke' --raw | audit2allow -M my-rhsmcertdworke # semodule -X 300 -i my-rhsmcertdworke.pp Additional Information: Source Context system_u:system_r:rhsmcertd_t:s0 Target Context system_u:object_r:root_t:s0 Target Objects virt.module [ file ] Source rhsmcertd-worke Source Path rhsmcertd-worke Port <Unknown> Host server.example.com Source RPM Packages Target RPM Packages Policy RPM selinux-policy-3.14.3-41.el8_2.5.noarch Selinux Enabled True Policy Type targeted Enforcing Mode Enforcing Host Name server.example.com Platform Linux server.example.com 4.18.0-193.13.2.el8_2.x86_64 #1 SMP Mon Jul 13 23:17:28 UTC 2020 x86_64 x86_64 Alert Count 265 First Seen 2020-06-24 13:55:40 EDT Last Seen 2020-09-22 13:35:09 EDT Local ID 97a1c0df-81ed-4c08-ba27-41c5067b713b Raw Audit Messages type=AVC msg=audit(1600796109.687:168): avc: denied { read } for pid=3912 comm="rhsmcertd-worke" name="virt.module" dev="dm-0" ino=50331783 scontext=system_u:system_r:rhsmcertd_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=file permissive=0 Hash: rhsmcertd-worke,rhsmcertd_t,root_t,file,read[ Readers also enjoyed:Accessing SELinux policy documentation ]
You can also use the command below for the raw audit messages, although they are significantly less helpful (and they are included in the sealert output).
[root@server ~]# grep "denied"/var/log/audit/audit.logHere the language is less user-friendly, and there are no suggested steps to take.
type=AVC msg=audit(1600796109.687:168): avc: denied { read } for pid=3912 comm="rhsmcertd-worke" name="virt.module" dev="dm-0" ino=50331783 scontext=system_u:system_r:rhsmcertd_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=file permissive=0Ghost denials
On special occasions (special, as in their ability to generate frustration), the SELinux AVC can deny a service without alerting the user that the denial occurred. When this happens, a little forensic digging is needed. Recently, my colleague, and writer extraordinaire, Ken Hess, installed a document management system on a CentOS virtual machine. During this installation, the DMS failed to install with no explanation. After a long, frustrating troubleshooting process, Ken discovered that there were repeated denial messages in /var/log/audit/audit.log.
type=AVC msg=audit(1602171971.334:438): avc: denied { write } for pid=12398 comm="httpd" name="31b32f0dfa1f50d778065b767582bb9b.js" dev="dm-0" ino=553885 scontext=system_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:httpd_sys_content_t:s0 tclass=file permissive=0Upon further inspection of /var/log/messages, he found the following:
Oct 8 12:33:17 dms python: SELinux is preventing /usr/sbin/httpd from name_connect access on the tcp_socket port 25.#012#012***** Plugin catchall_boolean (32.5 confidence) suggests ******************#012#012If you want to allow httpd to can network connect#012Then you must tell SELinux about this by enabling the 'httpd_can_network_connect' boolean.#012#012Do#012setsebool -P httpd_can_network_connect 1#012#012***** Plugin catchall_boolean (32.5 confidence) suggests ******************#012#012If you want to allow httpd to can sendmail#012Then you must tell SELinux about this by enabling the 'httpd_can_sendmail' boolean.#012#012Do#012setsebool -P httpd_can_sendmail 1#012#012***** Plugin catchall_boolean (32.5 confidence) suggests ******************#012#012If you want to allow nis to enabled#012Then you must tell SELinux about this by enabling the 'nis_enabled' boolean.#012#012Do#012setsebool -P nis_enabled 1#012#012***** Plugin catchall (4.5 confidence) suggests ***************************#012#012If you believe that httpd should be allowed name_connect access on the port 25 tcp_socket by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'httpd' --raw | audit2allow -M my-httpd#012# semodule -i my-httpd.pp#012 Oct 8 12:33:20 dms setroubleshoot: SELinux is preventing /usr/sbin/httpd from name_connect access on the tcp_socket port 25. For complete SELinux messages run: sealert -l ce75fc38-5696-4b21-b099-7780db5960f3Finally, something useful. He then ran the this command:
# sealert -l ce75fc38-5696-4b21-b099-7780db5960f3Which provided this information:
[user@host ~] # SELinux is preventing /usr/sbin/httpd from name_connect access on the tcp_socket port 25. ***** Plugin catchall_boolean (32.5 confidence) suggests ****************** If you want to allow httpd to can network connect Then you must tell SELinux about this by enabling the 'httpd_can_network_connect' boolean. Do setsebool -P httpd_can_network_connect 1 ***** Plugin catchall_boolean (32.5 confidence) suggests ****************** If you want to allow httpd to can sendmail Then you must tell SELinux about this by enabling the 'httpd_can_sendmail' boolean. Do setsebool -P httpd_can_sendmail 1 ***** Plugin catchall_boolean (32.5 confidence) suggests ****************** If you want to allow nis to enabled Then you must tell SELinux about this by enabling the 'nis_enabled' boolean. Do setsebool -P nis_enabled 1 ***** Plugin catchall (4.5 confidence) suggests *************************** If you believe that httpd should be allowed name_connect access on the port 25 tcp_socket by default. Then you should report this as a bug. You can generate a local policy module to allow this access. Do allow this access for now by executing: # ausearch -c 'httpd' --raw | audit2allow -M my-httpd # semodule -i my-httpd.pp Additional Information: Additional Information: [root@dms dms]# more sealert.txt SELinux is preventing /usr/sbin/httpd from name_connect access on the tcp_socket port 25. ***** Plugin catchall_boolean (32.5 confidence) suggests ****************** If you want to allow httpd to can network connect Then you must tell SELinux about this by enabling the 'httpd_can_network_connect' boolean. Do setsebool -P httpd_can_network_connect 1 ***** Plugin catchall_boolean (32.5 confidence) suggests ****************** If you want to allow httpd to can sendmail Then you must tell SELinux about this by enabling the 'httpd_can_sendmail' boolean. Do setsebool -P httpd_can_sendmail 1 ***** Plugin catchall_boolean (32.5 confidence) suggests ****************** If you want to allow nis to enabled Then you must tell SELinux about this by enabling the 'nis_enabled' boolean. Do setsebool -P nis_enabled 1 ***** Plugin catchall (4.5 confidence) suggests *************************** If you believe that httpd should be allowed name_connect access on the port 25 tcp_socket by default. Then you should report this as a bug. You can generate a local policy module to allow this access. Do allow this access for now by executing: # ausearch -c 'httpd' --raw | audit2allow -M my-httpd # semodule -i my-httpd.pp Additional Information: Source Context system_u:system_r:httpd_t:s0 Target Context system_u:object_r:smtp_port_t:s0 Target Objects port 25 [ tcp_socket ] Source httpd Source Path /usr/sbin/httpd Port 25 Host dms Source RPM Packages Target RPM Packages Policy RPM selinux-policy-3.13.1-252.el7.noarch Selinux Enabled True Policy Type targeted Enforcing Mode Permissive Host Name dms Platform Linux dms 3.10.0-1062.el7.x86_64 #1 SMP Wed Aug 7 18:08:02 UTC 2019 x86_64 x86_64 Alert Count 2 First Seen 2020-10-08 12:33:15 CDT Last Seen 2020-10-08 12:33:15 CDT Local ID ce75fc38-5696-4b21-b099-7780db5960f3 Raw Audit Messages type=AVC msg=audit(1602178395.253:461): avc: denied { name_connect } for pid=12565 comm="httpd" dest=25 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:smtp_port_t :s0 tclass=tcp_socket permissive=1 Hash: httpd,httpd_t,smtp_port_t,tcp_socket,name_connectWhen I asked Ken to weigh in on his experience, he had the following to say:
It's hard to know when an error, or lack of an error, is caused by SELinux. When you've checked firewall exceptions, filesystem permissions, and application prerequisites, your next step is to check SELinux context issues. The most common remedy is to disable SELinux by placing it into Permissive mode, which isn't a good idea because doing so leaves your system vulnerable. The better method is to check the /var/log/messages and the /var/log/audit/audit.log files for SELinux denials and work from there to individually remedy the denials until your application works. SELinux needs to remain in Enforcing mode to do this.
The troubleshooting list looks like the following when setting up a new application:
1. Check firewall exceptions for your application's ports.
2. Check filesystem permissions to ensure that your service account has the correct permissions to read, write, and execute where necessary.
3. Check your application's prerequisites and dependencies.
4. Check the /var/log/messages and /var/log/audit/audit.log files for SELinux denials.
SELinux Permissive mode can be used briefly to check if SELinux is the culprit in preventing your application from working. Once you've determined it to be the problem, return it to Enforcing mode and begin changing relevant contexts. See this information for SELinux contexts.
[ Improve your skills managing and using SELinux with this helpful guide. ]
Wrap up
SELinux is an effective security framework that can be incredibly useful when correctly implemented. As with any good security system, you will have denials over time. Knowing what to do with the information at your disposal is paramount to solving any issues that may be present or for allowing legitimate processes through the system. Now you know where the logs are kept and how to best interpret them. In the future, you'll be able to respond to any SELinux alerts that flag on your system.
Topics: Linux Linux administration Security
