FOR518 is the first non-vendor-based Mac and iOS incident response and forensics course that focuses students on the raw data, in-depth detailed analysis, and how to get the most out of their Mac and iOS cases. The intense hands-on forensic analysis and incident response skills taught in the course will enable analysts to broaden their capabilities and gain the confidence and knowledge to. UPDATE (April 4, 2018): Monitor.app now supports macOS 10.13. As a malware analyst or systems programmer, having a suite of solid dynamic analysis tools is vital to being quick and effective. These tools enable us to understand malware capabilities and undocumented components of the operating system.
Yaml artifact library file, so that other applications can use it. No crapy/fancy xml things. ForensicsWiki page, one of the point of reference for forensics practitioners. This way the effort is centralized and made only once. Mac OS X Forensics Artifacts. Here is the shared spreadsheet for the OSX artifacts. Working with NTFS – integrating Mac forensics in a Windows centric forensic lab; Review of Recommended Applications – our recommendations for commercial and non-commercial tools to assist with Mac forensics; Review of Automated Forensic Tools – our review of current automated Mac forensic.
Undocumented, unexplored, and underutilized, that is until now. Apple FSEvents or File System events are an invaluable artifact for every Apple examiner and should be a go to resource for artifacts relating to file system activity that occurred in the past. In this post I will be covering a high-level overview of Apple FSEvents logs that are stored to disk including background information and behavior of FSEvents, mainly dealing with OS X but also touching on iOS.
Topics of discussion include:
Introduction
FSEvents or File System Events are recorded events of changes to file system objects such as files and folders that occur on a volume mounted in OS X. It is used by several components of the OS including TimeMachine and Spotlight.
OS X 10.5 and 10.6 only captured events related to folders. Starting with OS X 10.7, file events were introduced. Similar to the NTFS change journal for Windows systems which records file system activity over time and stores the data in the UsnJrnl:$J, FSEvents records file system activity over time also and stores the data in FSEvent log files. When FSEvent logs are parsed, file system events that have occurred in the past such as file, folder, symbolic link and hard link creations, removes, renames, modifications, permission changes and more can be reviewed providing another great resource of forensic artifacts to the examiner. Information recorded in the logs that can be of value to an investigation include: mounting and unmounting external drives and disk images, activity within a users profile directory, document editing, internet activity, files moved to the trash, downloaded files, and much more. FSEvent Logs LocationOS X Location
On OS X, FSEvent logs are located in a hidden folder named ‘.fseventsd/’ in the root directory of the system partition. Note that by default, this folder is hidden from view within Finder on a running Mac and accessing it requires elevated privileges.
iOS Locations
For iOS devices there are multiple locations where FSEvent logs are stored. Note that jailbreaking a device is often required to pull FSEvent logs. FSEvent log locations include:
1) System: /.fseventsd’
2) Data: /private/var/.fseventsd’
a. Usage after the device was first used by a user.
3) Developer Patch:/DeveloperPatch/.fseventsd’
a. Contains records prior to the devices first use by a user. This might also include patch activity if the OS was patched after being purchased, but this has not been verified.
External Device Locations
For external devices that were plugged into a Mac, when they exist they will be located in the root folder of the device within the “.fseventsd/” directory. There are several scenarios however that will result in an external device having a “.fseventsd/” folder, but no logs within the folder:
a. An external device was plugged in to a Mac, but the device was unplugged without safely removing it resulting in FSEvent logs being lost. For example, the user pulls a usb drive from a Mac without using the unmount icon in Finder.
b. An external device was plugged in to a Mac, the device was safely removed (unmounted), but the device was unplugged before the OS could finalize writing FSEvents to disk.
c. An external device was plugged in to a Mac, but there is an unknown file system compatibility issue with the device.
FSEvent Log Files
The ‘.fseventsd/’ directory can contain anywhere from one log file, a few, or even as many as hundreds each of which contain historical file system and user activity over the course of a few days to months.
An individual log file can span multiple days depending on how much activity was occurring on the volume. System updates, upgrades, and application installs tend to generate a large number of FSEvents.
The logs are gzip archives and are named using a specific naming standard which is represented in hexadecimal format. The name of each FSEvent log refers to the last Event ID stored in the FSevent log file plus 1.
For example, using the first FSEvent log listed in the figure below, when the file name is converted from the hex value of ‘00000000000a4b3e’ to decimal, the value is 674,622. Therefore, the last Event ID within this file is 674,621 which is 1 less than 674,622.
FSEvent Record Lifecycle
FSEvent records are initially stored in memory. When a change occurs to an object on a volume, the FSEvents API checks to see if the object has already been assigned an Event ID using the relative full path of the object. Components of an FSEvent Record including Event ID’s will be discussed in more detail later. If no event ID has been assigned for the object in memory, one will be assigned to it and the relative full path of the object, the record flags, and the event ID will be stored in memory. If the object was already assigned an event ID, the API will update the record flags to include the current change. For each subsequent change that the object incurs while it has been assigned an event ID in memory, the API coalesces the changes (the record flags) and stores it as a single event for the object.
When the FSEvents API has determined that the memory buffer is full or the volume is being unmounted it flushes the events to disk in the form of FSEvent log files.Once flushed to disk, the API will not modify the contents of the logs.
As previously mentioned, multiple changes can be recorded in a single FSEvent record, each of which indicates what kind of changes occurred to the object. If for example, a text file on the user’s desktop was created, then modified, five minutes later modified again, and then finally removed (deleted) the two modifications will only be recorded once as a single event. The event record for the text file when parsed might look something like the following:
Note that even though the FSEvent record above shows what kinds of changes occurred, the order of changes are not accounted for within the logs.
These limitations are the result of the lack of granularity imposed by the FSEvents API when recording changes and not the result of parsing what was recorded. FSEvent Log Format
FSEvent logs are stored as compressed archive files in gzip format. Once the files are uncompressed they can be opened using a hex editor so that raw strings can be viewed.
FSEvent Record Components
Each FSEvent log contains records that represent historical changes to objects on a volume. Each record within an FSEvent log consists of three major components:
* Update:
With the release of MacOS High Sierra (10.13), each record now consists of four components:
MacOS High Sierra 10.13 FSEvent Record Components
Record Full Path
The Record Full Path is the unique relative full path to the file system object that incurred a change.
Record Event IDs
According to Apple documentation, event IDs are “monotonically increasing per system, even across reboots and drives coming and going. They bear no relation to any particular clock or timebase.” [1]
The Full Path to a file system object is only assigned one Event ID per FSEvent log. However, it can have multiple Event IDs across multiple logs. Record Event Flags
Event Flags are stored in each FSEvent record and contain bit flags that indicate:
Record FS Node ID
With the release of MacOS High Sierra, the FSEvent records now include a File System Node ID or as Apple refers to it, a File ID Key. [2]. It is the File System object inode number. If for example, the file system was HFS+, this would represent that Catalog Node ID.
Looking at the Hex
The example below is of an unzipped FSEvents log from a thumb drive containing activity that occurred on the drive. This structure applies to MacOS 10.12 and lower.
The first twelve bytes of the log are allocated to the FSEvents log file page header and begins with a magic number of “1SLD”.
*Update: Taking into the byte order of little-endian, “1SLD” actually reads “DLS1” or Disk Logger Stream version 1. Thanks go out to Joachim Metz for pointing that out.
There can be multiple page headers in one FSEvents log. Following each page header is the start of a Record Full Path of an event.
In the figure below, there are three events in total but only the first will be discussed. The first stored event has a Record Full Path of “My_File_1.txt” at offset 0x0C.
The Null Terminator “0x00” which is at the end of every Record Full Path indicates that it is the end of the full path.
Following the full path’s null terminator is the record’s Event ID. For the first event in our example this is located at offset 0x1a. The Event ID for this record is 0x01548d or 87,181 in decimal. After the Event ID are the Record Flags which is stored at offset 0x22 for this event. Decoding record flags will not be covered in great detail but for this event the hex value of 0x5505800 indicates that this is a file that has been created, modified, finder information changed, and inode metadata changed.
*Update: With the release of MacOS 10.13, the FSEvent record structure now includes an additional value. It is the File System Node ID.
MacOS High Sierra 10.13
For MacOS High Sierra FSEvents, the record structure is different from previous version of MacOS. It now contains the FS Node ID which follows the Reason flags.
Note that there can be instances where the fullpath is not reported for an event. It is currently unknown why. An example of this is shown in the above example as the first event. This has been observed in FSEvents for all MacOS releases.
FSEvent Record Flags
In the table below information about each of the possible flags is covered including but not limited to possible scenarios that would cause this flag to be set.
Parsing FSEvents
There are both paid and open-source tools available that parse FSEvents.
———————————-
Usage: FSEParser_v2.1.py -c CASENAME -s SOURCEDIR -o OUTDIR
Options:*
-h, –help show this help message and exit
-c CASENAME The name of the current session, used for naming standards
-s SOURCEDIR The source directory containing fsevent files to be parsed
-o OUTDIR The destination directory used to store parsed reports
Once the command is issued, the parser will extract the records contained in each of the FSEvent log files located in the SOURCEDIR supplied in the command, placing the parsed data into a tab delimited tsv file and an SQLite database located in the OUTDIR supplied.
FSEventsParser Output
When the parser has finished parsing, the tab delimited file can by opened using Excel. FSEvent records are stored in the log in alphabetical order using the record full path (filename column in the screenshot below). To achieve a more chronological sort order in the parsed output, sort by the “event_id” column as ascending. Keep in mind however that the chronological order relates only to when the first change occurred to an object.
The FSEventsParser script also produces an SQLite database containing parsed records which is useful for times when parsed records can be in excess of 1 million. The SQLite database also comes in handy when running queries to narrow down the records of interest.
Interesting Events in OS X
All events discussed were parsed using G-C Partners’ FSEventParser tool which output events into an SQLite database. Events listed here only scratch the surface of what can be found in FSEvents logs. This section is provided to give you an idea of what can be found within those logs. Interpreting the exact cause as to how these events were generated requires additional testing and validation.
Trash Activity
FSEvents log files record Trash activity which includes files sent to the Trash and Trash emptying. Files that were sent to the Trash will include the mask “Renamed”. Files that were emptied from the Trash will include the mask “Removed”.
The image below shows files and folders that were sent to the Trash. Note that in the Mask column, all events include the word “Renamed”. Note that “Renamed” is somewhat of a misnomer. A “Renamed” mask is assigned to an event in two ways:
In the case of the Trash folder, when files are sent to the Trash they receive a “Renamed” mask because they were moved to the Trash from some other location on the volume.
SQLite Query Example
SELECT
*, _ROWID_ “NAVICAT_ROWID” FROM “fsevents” WHERE “filename” LIKE ‘Users/%/.Trash/%’ System Boots
When a system is booted, the OS mounts “/home” and “/net”. We can query the SQLite database to get a list of approximate dates that the system was booted. In the image below, only “/home” was queried.
SQLite Query Example
SELECT
*, _ROWID_ “NAVICAT_ROWID” FROM “fsevents” WHERE ( “filename” = ‘/home’ OR “filename” = ‘/net’ ) AND “mask” LIKE ‘%mount%’ User Profile Folders
Activity within the User’s profile can also be of particular interest. This can include documents, downloads, and desktop activity. In the image below, a folder was renamed and two DMGs were created on the Desktop. In the Downloads directory, a ZIP file was created.
Note that the creation of some of these files was initiated by the user (me), and some were created by an application or the OS. In the Downloads directory, .BAH.XSEHo, .DS_Store, and M9hlK5RI.zip.part were not created nor modified by me directly.
SQLite Query Example
SELECT
*, _ROWID_ “NAVICAT_ROWID” FROM “fsevents” WHERE ( “filename” LIKE ‘Users/%/Documents/%’ OR “filename” LIKE ‘Users/%/Downloads/%’ OR “filename” LIKE ‘Users/%/Desktop/%’ ) AND “filename” NOT LIKE ‘Users/%/Library/Caches/com.%’ AND “filename” NOT LIKE ‘Users/%/Library/Containers/com.%’ AND “filename” NOT LIKE ‘Users/%/Documents/Microsoft User Data/Office %’ Mounted Volumes
When a drive is plugged in, the OS will try to mount it. When a DMG is double clicked, the OS will try to mount it. Mounts are also recorded within FSEvents.
SQLite Query Example
SELECT
*, _ROWID_ “NAVICAT_ROWID” FROM “fsevents” WHERE “mask” LIKE ‘%mount%’ AND “filename” NOT LIKE ‘/net’ AND “filename” NOT LIKE ‘/home’ Internet Activity
Web browsers such as Safari and Chrome store website addresses or URLs in the name of a files associated with internet activity. The changes to those files are recorded in FSEvents. Most of the websites listed in the image below were the result of me directly visiting the site, others appear to have been from third party sites not directly visited.
SQLite Query Example
SELECT
*, _ROWID_ “NAVICAT_ROWID” FROM “fsevents” WHERE “filename” LIKE ‘Users/%/Library/Caches/Metadata/Safari/History/%’ OR “filename” LIKE ‘Users/%/Library/Application Support/Google/Chrome/Default/Local Storage/%’ OR “filename” LIKE ‘Users/%/Library/Safari/LocalStorage/%’ Interesting Events in iOSiCloud Synced Files
File synced from a connected iCloud account are recorded by FSEvents.
SQLite Query Example
SELECT
*, _ROWID_ “NAVICAT_ROWID” FROM “fsevents” WHERE “filename” LIKE ‘mobile/Library/Mobile Documents/com~apple~CloudDocs/%’ Internet Activity
Internet activity on a mobile device is also recorded.
SQLite Query Example
SELECT
*, _ROWID_ “NAVICAT_ROWID” FROM “fsevents” WHERE “filename” LIKE ‘%websitedata/local%’ 2018 Forensic App Mac OsEmail Activity
Email activity including received items (inbox), sent items, and associated attachment names are also recorded by FSEvents.
2018 Forensic App Mac DownloadSQLite Query Example
SELECT
*, _ROWID_ “NAVICAT_ROWID” FROM “fsevents” WHERE “filename” LIKE ‘mobile/Library/Mail/%’ Caveats
Like any forensic artifact there are caveats the one must be familiar with. For FSEvents these include:
Lost FSEvent Logs
There are several scenarios that will result in FSEvent logs becoming lost or removed (deleted) from the volume. This can include:
On OS X, when FSEvent logs are removed from a volume, they become unallocated. Therefore, it is possible to carve for FSEvent logs using data recovery techniques and carving tools to carve for gzip files.
Lack of Timestamps
As previously mentioned, FSEvent records consist of three components: the full path, record flags, and the event ID. Note that a timestamp is not one of the three components.
There are ways to overcome this but so far none will give you precise timestamps for when an event occurred. Temporal data can be pulled from the names of Apple system log files and potentially the names of other files that are recorded within FSEvent logs to help determine the approximate time that an event occurred.
External Devices
FSEvents on external devices can be especially volatile.
Anti-Forensic Measures
According to Apple documentation, it is possible to disable FSEvents on a volume. The existence of a file named “no_log” in the .fseventsd folder is a tipoff that FSEvents has been disabled for a volume. In the wild and in testing, this was never the observed case. [2]
Coalescing of Multiple Changes
FSEvent record flags can indicate that multiple changes occurred to a file or folder. Due to the nature of how the FSEvents API records and stores changes, the granularity required to determine in what sequence each individual change occurred and how many times those changes occurred is not possible.
At best, using FSEvents, we can only determine the sequence of when the first change occurred for an objected.
The bad news is that there is a major issue to be mentioned here. Remember that FSEvents stores changes primarily based off of the relative full path of a file system object. So, lets say for example that I created a file on my Desktop named “example.zip” . Within a short period of time, this file became deleted in whatever fashion, then another file with the exact same name and location is created again. I now how an entirely different file on my desktop. The FSEvents API, sees them both as the same file because it only takes into account the relative full path when recording changes. So even though “example.zip” was created twice and is two completely different files it is only recorded once (keeping in mind that this occurred within a short period of time).
The good news is that FSEvents does record the fact that a file/folder was changed, and we know what kinds of changes occurred. Just not the order and frequency.
References
[1] https://developer.apple.com/reference/coreservices/fseventstreameventid
[2] https://developer.apple.com/documentation/coreservices/kfseventstreameventextendedfileidkey?language=objc
[3] https://developer.apple.com/library/content/documentation/Darwin/Conceptual/FSEvents_ProgGuide/FileSystemEventSecurity/FileSystemEventSecurity.html#//apple_ref/doc/uid/TP40005289-CH6-SW5
Network Scanning Overview
So far, the Automated Network Scanning Team ! has learned about Python and Nmap. How delete docker app on mac. We are planning to use Python to create an automated network scanner and report generator with Nmap. To do this, we had to learn how to install various Python packages, such as libnmap, a package that enables the execution of Nmap. We also installed argparse, which allows users to change how a program runs without changing the code of the program, and smtplib, which allows Python to send emails from an SMTP server.
Python
The Python scanning portion will first take inputs from the command line using the argparse package, allowing the user to run it with different inputs, specify target IP’s, and identify the output destination of the email report. The team added these features to the program. Which was intended to run remotely from a Raspberry Pi integrated into a network. After the user specifies these parameters, the program launches a scan utilizing the libnmap package. Unlike all the other packages, to install libnmap we had to learn how to use pip, a python package installation tool. This was a new experience for the team, but we successfully learned how to install packages using pip.
After the scan completes, it is re-organized for readability, and then the smtplib package is used to send the results in .csv file to the target destination. Throughout this process, we had to learn from the documentation of all three of these packages, which we had never worked with before. It improved our understanding of the Python language and sharpened our programming skills.
Network Mapper – Nmap
While using Nmap, we studied different types of scans to obtain all the information needed to compile a full report. We began by examining a ping scan, which scans through a range of IP’s for promising IP addresses to scan. A ping scan uses the same packets as a standard ping request. This scan was done first in our program in order to discover any viable hosts that were up and running, while being relatively fast compared to other host discovery scans. It also provided enough information to execute our next scans. The next scan that ran was the OS Fingerprinting scan.
This scan sent packets to a host, then ran dozens of tests on that host. After this, Nmap compared the results to a database of more than 2,600 OS fingerprints, trying to find a good match. We used this scan type to gather additional information on the target hosts/network.
Conclusion
One added feature of our automated network scanner is to find known vulnerabilities. We decided to scan for a cryptographic vulnerability, the Heartbleed Bug. This vulnerability allows the stealing of information encrypted with OpenSSL, a popular encryption protocol. There are Nmap scripts designed to scan for targets that are vulnerable to this bug. Nmap maintains a database of scripts that other users have found useful for security applications, and individual users can expand their nmap abilities by scripting to obtain new and different information.
Throughout the past few weeks, our Automated Network Scanning Team has learned about several Python packages, like libnmap, argparse, and smtplib. We have explored the functionalities of Nmap, bringing both together in our Python automated network scanner.
To learn more about this and other blogs of the LCDI visit us here: LCDI Blog.
Stay in the loop on our current and upcoming projects and events by following us on Facebook, Twitter, or Instagram.
Comments are closed.
|
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |