LPI E - System Data

LPI E - Scripting

4.3 Where Data is Stored

Review of Topics:
Various types of information stored on a Linux system
Part 1 of 2:  Part 2 - A Extensive Look

• Programs
• Configuration
• Processes
• Memory addresses
• System messaging
• Logging 
• Using the tools
• ps
• top
• free
• syslog - Debian or Fedora
  
• dmesg - Debian or Fedora
  
• and by locating information from directories like 
• "/etc/"
• "/bin"
  
• "/var/log/" Debian or Fedora
  
• "/boot/"
• "/proc/"
• "/dev/"
• "/sys/"

I'll also provide a reference to some other tools at the end

1. Finding information about Linux Programs and Configuration:

   Programs in Linux are typically stored in directories like
   • "/bin/"
   • "/sbin/"
   • "/usr/bin/"
   • "/usr/sbin/
   • "bin" meaning the executable binaries
    
   
   To locate information about a program, you can use the "which" command followed by the name of the program
2. • $ which ls
   • This will display the full path to the program.
   • To check if a command is available in the system's PATH
     
   • $ which -a date
     
   • To find the location of an executable binary file:
   • $ which -a -x python
   • /usr/bin/python
   • /usr/bin/python2.7
   • /usr/bin/python3.6
     
   • To list all directories in the system's PATH. This will print a list of directories separated by colons, which represent the directories in the PATH environment variable.
     
   • $ which -a -p ''
     
    
3. To get more information about a program you can use the "man" command followed by the name of the program
   • This will display the manual page for the program, which includes a description of the program's purpose, how to use it, and other useful information.
   • $ man ls
     
   • To search for all manual pages related to "network", you would type
   • $ man -k network
   • To view the manual page for chmod in section 2 (system calls), you would type
   • $ man 2 chmod
   • To view a brief summary of the ls command, you would type
   • $ man -f ls
   • To view the manual page for ls as a plain text file, you would type
   • $ man -P cat ls
   • These are just a few of the basic parameters for using the man command in the Linux Terminal. There are many more options available for advanced usage.
    
4. Here are just a few examples of the many parameters and switches that can be used with the "find" command to locate configuration files for Linux programs and binaries.
    
   By understanding these basic parameters, a Linux learner can begin to effectively search for files and directories using the terminal. 

   👇👽💦

   https://www.certificationmethods.com/2023/03/find-secrets.html
   

5. Finding information about Linux Processes:

   The "ps" command
   
   
   
6. The "ps" command can be used to display information about running processes. By default, it shows information about processes running in the current terminal session. 
   • To display information about all processes, use the "-e" option, e.g., 
   • $ ps -e
   • To display information about all running processes, type the following command in the terminal
   • $ ps aux
     
   • To display a list of all running processes along with their parent process ID, type the following command:
   • $ ps -ef
     
   • To display a list of all running processes sorted by their CPU usage, type the following command
   • $ ps aux --sort=-%cpu
     
   • To display a list of all running processes owned by a specific user, type the following command, replacing "username" with the actual username
   • $ ps -u username
     
   • To display a tree view of all running processes, type the following command
   • $ pstree 

The "top" command

• The "top" command can be used to display real-time information about processes, including 
• CPU usage
• memory usage
• statistics
• monitoring system performance
• NOTE: It updates the information continuously
  

• To run top with a specific refresh rate, use the -d parameter followed by the number of seconds between refreshes. For example, to refresh every 5 seconds
• $ top -d 5
• To sort the process list by a specific column, use the -o parameter followed by the column name. For example, to sort by memory usage
  
• $ top -o RES
• To filter the process list by a specific user, use the -u parameter followed by the username. For example, to only show processes owned by the user "jdoe"
• $ top -u jdoe
• To kill a process from within top, first select the process using the arrow keys or the PID number. Then, press the k key and enter the PID when prompted. For example, to kill the process with PID 1234, select the process in top, press k, and enter 1234
• # Select the process in top using the arrow keys or PID number
• k
• 1234
  
• To change the priority of a process from within top, first select the process using the arrow keys or the PID number. Then, press the r key and enter the new priority when prompted. 
• For example, to set the priority of the process with PID 1234 to -10, select the process in top, press r, and enter -10
• # Select the process in top using the arrow keys or PID number
• r
• -10
• To search for a specific process name within top, press the / key followed by the process name. For example, to search for processes with the name "apache", press / and enter "apache".
• /apache

1. Finding information about Linux Memory Addresses:

   • The "free" command can be used to display information about system memory usage, including the amount of free and used memory. 
   • By default, it displays the information in kilobytes, but you can use the "-m" option to display it in megabytes, e.g., "free -m".
   • $ free
   • $ free -m
     
   • The "-h" switch will display the memory information in a more human-readable format, showing memory sizes in units like "MB" and "GB" instead of bytes
     
   • Display memory information in human-readable format
   • $ free -h
   • Display memory information in a continuous stream
   • The "-s" switch will display the memory information in a continuous stream, with a specified delay between each update. In this example, the delay is set to 1 second
   • $ free -s 1

   Other common tools for memory information

   • Check memory usage of a specific process with pmap command, This will display the memory usage of the process with the given process ID (<pid>).
     
   • pmap <pid>
     
   • Monitor memory usage of a specific process in real-time with watch command, This will display the memory usage of the process with the given process ID (<pid>) every second
     
   • watch -n 1 pmap <pid>
   • Analyze memory usage with vmstat command, This will display the memory usage statistics in a detailed format, including information like the number of pages swapped in and out
     
   • vmstat -s

2. Finding information about Linux System Messaging and Logging:
   

   In the Debian Linux terminal, dmesg and syslog are two different commands used to access different types of system logs.

   👇👽💦

       dmesg: This command is used to display the kernel ring buffer messages. These messages are generated by the kernel during the system boot and other events, and they are stored in a circular buffer in memory. The dmesg command can be used to display the contents of this buffer, which contains information about hardware detection, device drivers, and other kernel-level events.
   
       syslog: This command is used to display the system log messages generated by various system services and applications. These messages are stored in log files located in the /var/log/ directory on most Linux distributions. The syslog daemon is responsible for collecting and storing these messages, which are usually categorized by severity level (debug, info, warning, error, etc.) and source (kernel, system services, user applications, etc.).

1. System messages and logs are typically stored in directories like "/var/log/" for Debian uses journalctl for log file access instead of direct file read.
   
   • This will display kernel messages with a severity level of "info".
   • Debian: $ syslog -k info
   • Fedora: $ journalctl | grep "info"
     
   • To view the entire syslog
   • Debian: $ cat /var/log/syslog
   • Fedora: $ journalctl
     
   • To view the last 100 lines of the syslog
   • Debian:$ tail -n 100 /var/log/syslog
   • Fedora:$ journalctl -n 100
     
   • To follow the syslog in real-time
   • Debian: $ tail -f /var/log/syslog
   • Fedora: journalctl -f
     
   • To search the syslog for a specific string
   • Debian: $ grep "search string" /var/log/syslog
   • Fedora: $ journalctl | grep "search string"
     
   • This command will display kernel messages related to USB devices connected to the system. You can replace "usb" with the name of any other device to view kernel messages related to that device.
   • Debian: $ dmesg | grep -i usb
   • Fedora: $ journalctl | grep -i usb
     
   • Debian:This command will display kernel messages generated between March 1, 2022 and March 5, 2022. You can modify the date range as needed.
   • Fedora: This command will display kernel messages generated between March 1, 2022 and March 5, 2022. The _TRANSPORT=kernel option limits the output to only kernel messages.
     
   • Debian: $ dmesg --since "2022-03-01" --until "2022-03-05"
   • Fedora: $ journalctl --since "2022-03-01" --until "2022-03-05" _TRANSPORT=kernel
     
   • Debian: This command will display the kernel log in real-time, continuously updating the log as new messages are generated. The -w option enables the command to wait for new messages, and -H option formats the output with human-readable timestamps
   • Debian: This command will display the kernel log in real-time, continuously updating the log as new messages are generated. The -f option enables the command to wait for new messages. 
   • Fedora: This command will display the kernel log in real-time, continuously updating the log as new messages are generated. The -f option enables the command to wait for new messages.
     
   • Debian: $ sudo dmesg -wH
   • Fedora: $ sudo journalctl -f
     
   • Debian: This command will save the kernel log to a file named kernel_log.txt. You can open the file in a text editor to view the log messages later.
   • Fedora: This command will save the kernel log to a file named kernel_log.txt. The -b option limits the output to messages generated during the current boot. Display kernel log messages in reverse chronological order:
     
   • Debian: $ dmesg > kernel_log.txt
   • Fedora: $ journalctl -b > kernel_log.txt
     
   • Debian: This command will display kernel log messages in reverse chronological order, with the most recent messages appearing first. The -r option sorts the messages by their severity level.
   • Fedora: This command will display kernel log messages in reverse chronological order, with the most recent messages appearing first. The -k option limits the output to kernel messages, and the -r option sorts the messages by their severity level.
     
   • Debian: $ dmesg -r
   • Fedora: $ journalctl -k -r
     

2. Locating information from other directories:

   The "/boot/" directory contains files related to system booting, including the kernel and bootloader configuration files. 
   

   👇👽💦

   • This command will list the files and directories in the /boot/ directory. In Debian and Fedora, this directory typically contains the kernel image files, as well as the boot loader configuration files.
   • $ ls /boot/
   • This command will display information about the kernel image file. The $(uname -r) command substitution will return the current kernel release version, which will be appended to the vmlinuz- prefix. This command can be used to determine the architecture and other properties of the kernel image file.
   • $ ls /boot/grub2/
     
   • This command will list the files and directories in the boot loader configuration directory. In Fedora, the boot loader is typically GRUB 2, which uses the /boot/grub2/ directory for its configuration files.
   • $ cat /boot/grub2/grub.cfg
     
   • This command will display the contents of the GRUB 2 configuration file. This file contains the boot menu entries, which specify the kernel image file and any additional boot parameters.
   • $ cat /boot/grub2/grub.cfg
     
    
   "/etc/":
   This directory contains configuration files for various programs and services on your system. 
    
   you can find network configuration files in "/etc/network/" and Apache web server configuration files in "/etc/apache2/"
   

   👇👽💦
   

   These directories contain important configuration files for the system, so it's important to understand their contents and be careful when making changes to them.
   

   
   
   • This command will list the contents of the /etc/ directory in long format, including the file permissions, owner, group, and size. This can be useful for verifying the configuration files in the directory and ensuring that the correct permissions are set.
   • $ ls -l /etc/
     
   • This command will list the contents of the /etc/network/ directory in long format, including the file permissions, owner, group, and size. This directory contains the network configuration files for the system, such as interfaces and if-up.d/.
   • $ ls -l /etc/network/
   • This command will open the hostname file in the Nano text editor with superuser privileges. You can modify the contents of the file as needed, and then save and exit the editor to apply the changes. This file specifies the hostname of the system.
   • $ sudo nano /etc/hostname
   • This command will open the interfaces file in the Nano text editor with superuser privileges. You can modify the network configuration settings in the file as needed, and then save and exit the editor to apply the changes. This file specifies the network interfaces and their configuration.
   • $ sudo nano /etc/network/interfaces
     
   • This command will display the contents of the fstab file in the terminal. This file contains the filesystem mount information for the system.
   • $ cat /etc/fstab
     
   • This command will display the contents of the interfaces file in the terminal. This file specifies the network interfaces and their configuration.
   • $ cat /etc/network/interfaces
   
   
   "/var/log/":
   This directory contains log files generated by various programs and services on your system. The log files can be used to troubleshoot issues and monitor system performance.
    
   Some common log files include "/var/log/syslog" for system messages and "/var/log/auth.log" for authentication-related messages.
   

   👇👽💦

   • This command will list the contents of the /var/log/ directory in long format, including the file permissions, owner, group, and size. This can be useful for verifying the log files in the directory and ensuring that the correct permissions are set.
   • $ ls -l /var/log/
     
   • This command will list the contents of the /var/log/syslog/ directory in long format, including the file permissions, owner, group, and size. This directory contains the system log files for the system.
   • $ ls -l /var/log/syslog/
     
   • This command will display the contents of the auth.log file in the terminal. This file contains authentication-related messages, such as user logins and password changes.
   • $ cat /var/log/auth.log
     
   • This command will open the syslog file in the Nano text editor with superuser privileges. You can modify the contents of the file as needed, and then save and exit the editor to apply the changes. This file contains system log messages.
   • $ sudo nano /var/log/syslog
     
   • Debian: This command will open the messages file in the Nano text editor with superuser privileges. You can modify the contents of the file as needed, and then save and exit the editor to apply the changes. This file contains general system log messages.
   • Fedora: the syslog messages are stored in the /var/log/messages file, unlike in Debian-based distributions where they are stored in /var/log/syslog. Using the sudo command before the vi command will give you the necessary permissions to modify the file.
     
   • Debian: $ sudo nano /var/log/syslog/messages
   • Fedora: sudo vi /var/log/messages
     
   • Debian: This command will search for the string "error" in the syslog file and display any matching lines in the terminal. This can be useful for troubleshooting issues with the system.
   • Fedora: the system logs are stored in /var/log/messages instead of /var/log/syslog. The sudo command is used to execute the grep command with elevated privileges, allowing it to search the contents of the log file.
     
   • Debian: $ grep "error" /var/log/syslog
   • Fedora: $ sudo grep "error" /var/log/messages
     
   
   
   "/proc/":
   This is a virtual filesystem that contains information about system processes and hardware. You can find detailed information about running processes in "/proc/[pid]/" where [pid] is the process ID. 
    
   For example, you can view the process command line arguments in "/proc/[pid]/cmdline".

   👇👽💦
   

   Note that modifying the contents of the /proc/ directory can have serious consequences on the system, and should only be attempted by experienced users or administrators who fully understand the risks involved.
   
   
   
   • This will display a list of subdirectories, each representing a different system process or component.
   • $ ls /proc/
     
   • The /proc/[pid]/ directory contains information about a specific process, where [pid] is the process ID number. To display information about a specific process, use the cat command to read the contents of the /proc/[pid]/status file: This will display various information about the process, such as its process ID, parent process ID, and memory usage.
   • $ cat /proc/[pid]/status
     
   • You can modify the priority of a running process by changing the value of its nice value using the /proc/[pid]/ directory. The nice value determines the scheduling priority of a process, with lower values indicating higher priority. To modify the nice value of a process, use the echo command to write a new value to the /proc/[pid]/ directory: Replace [new_nice_value] with the desired new value for the nice priority. Note that you must have sufficient privileges to modify process priorities.
   • $ echo [new_nice_value] > /proc/[pid]/priority
     
   • The /proc/[pid]/cmdline/ directory contains the command line arguments that were used to start a specific process. To display the command line arguments for a running process, use the cat command to read the contents of the /proc/[pid]/cmdline file
   • $ cat /proc/[pid]/cmdline
   • This will display the environment variables as a series of null-separated strings. Similar to the previous example, you can use the tr command to replace null characters with spaces for better readability:
   • $ cat /proc/[pid]/cmdline | tr '\0' ' '
     
   • The /proc/[pid]/environ directory contains the environment variables that were set when a specific process was started. To display the environment variables for a running process, use the cat command to read the contents of the /proc/[pid]/environ file:
   • $ cat /proc/[pid]/environ | tr '\0' '\n'
     
   • The /proc/stat file provides information about system CPU usage, including the number of CPU cores, the amount of time spent in user, system, idle, and other modes, and more. To display this information, use the cat command to read the contents of the /proc/stat file. This will display a long string of numbers, with each line representing a different CPU core or statistic.
     
   • $ cat /proc/stat 
    
    "/dev/":
   This directory contains device files that represent physical and virtual devices on your system. You can use these files to interact with the devices using system calls. 
    
   For example, the "/dev/sda" file represents the first SCSI hard disk on your system.

   👇👽💦
   

   • This command will display a list of all device files in the /dev/ directory. You can use the -l option to display more detailed information about each file, such as file size and permissions.
   • $ ls /dev/
     
   • This command will change your current directory to the block device file for the first hard disk (sda). You can then use other commands to interact with the device, such as dd to make a backup of the disk.
   • $ cd /dev/sda
     
   • This command will display the raw data on the second hard disk (sdb). You can use this command to check the status of a device or to read data directly from a device.
   • $ cat /dev/sdb
     
   • This command will write one megabyte of zeros to the third hard disk (sdc). You can use this command to write data directly to a device, which can be useful for testing or debugging purposes.
   • $ dd if=/dev/zero of=/dev/sdc bs=1M count=1
     
   • This command will create a new character device file called mydevice with major number 10 and minor number 20. You can use this command to create new device files for custom hardware devices or for testing purposes. Note that you need to have root privileges to create new device files.
   • $ sudo mknod /dev/mydevice c 10 20
   
   
   "/sys/":
   This is a virtual filesystem that contains information about the system hardware and configuration. 
    
   You can find information about system devices, such as their drivers and status, in "/sys/class/" and "/sys/bus/" directories.

   👇👽💦

   • This will show a list of device classes, such as "block", "input", "net", etc. To list the devices within a specific class, use the class name as a subdirectory. For example: This will show a list of block devices, such as hard disks, SSDs, and USB drives, along with their corresponding device drivers.
     
   • $ ls /sys/class
   • $ ls /sys/class/block
     
   • This will show a list of buses, such as "pci", "usb", "scsi", etc. To list the devices within a specific bus, use the bus name as a subdirectory. For example: This will show a list of PCI devices, along with their corresponding device drivers.
   • $ ls /sys/bus
   • $ ls /sys/bus/pci/devices
     
   • This will display the current operational state of the "eth0" network interface, such as "up" or "down".
   • $ cat /sys/class/net/eth0/operstate
     
   • This will display the device driver loaded for the first sound card ("card0") in the system.
   • $ readlink /sys/class/sound/card0/device/driver
     
   • This will display the hardware configuration of the USB device connected to bus "1" and port "3" of the USB hub.
   • $ cat /sys/bus/usb/devices/1-1.3/device/uevent
     

 

Conclusion

Linux provides a rich set of tools and directories that can be used to find information about system processes, configuration, and performance. By using tools you can gain a deeper understanding of how your system is working and troubleshoot issues as they arise. Additionally, other useful tools can be used to further enhance your troubleshooting and monitoring capabilities.

These include:

• systemctl - Used to manage system services
• journalctl - Used to view system log messages
• ifconfig and ip - Used to manage network interfaces and configurations
• iptables - Used to manage firewall rules
• ssh - Used to securely connect to remote systems

By mastering these tools and techniques, you'll be well on your way to becoming a skilled Linux administrator. Remember to always reference the appropriate documentation for each tool or configuration file, as this will help ensure that you're using them correctly and effectively.