Pcap Formats

How the packets are encoded
4 min　|　 Ross Jacobs　|　 July 7, 2019

Table of Contents

Quicklinks: captype manpage | code

Capture Formats

Background

The internet is a testament to our ability to put aside our differences and agree to standards like Ethernet and TCP/IP. In that spirit of cooperation and interoperability, most network vendors have their own proprietary capture formats.

Format Prevalence Today

The majority of captures that you will deal with today are pcap or pcapng. With the prevalence of linux, libpcap, tcpdump, and Wireshark in network devices, most vendors now support the pcap-type natively or produce a hexdump that can be converted.

This pie chart is based on 6,734 captures from PacketLife, Wireshark Samples, and Wireshark Bugzilla (2019). Gzipped versions of capture types are considered that capture type. Each other capture type constituted < 1%.

pcap

Pcap as a format was born at the same time as tcpdump/libpcap which used it. Technically, this would place place it at 1988 when tcpdump was created. However, I think it’s fairer to place it at 1999 when tcpdump.org was launched and became more well-known.

Pcap is the most common capture type because libpcap has had support and been around for more than 20 years. As an older format, it allocates fewer fields for packet and capture metadata.

pcapng

Pcapng is an evolution from the pcap format, created to address some of its deficiencies. Namely, the lack of extensibility and inability to store additional information. Any file that uses comments MUST be a pcapng file because this is one of the features pcapng format enables.

For deconstructing pcapng structure, I would consult Sam’s Browne’s wonderful article on the subject.

Listing Available Formats

The full list of formats that your system supports can be found with tshark -F. A sample listing is available if you’re curious.

Captype

Capytpe reads a file and prints the file type. It has no flags and takes one or more files as argument.

Captype Example

$ captype testdir/*
literally_an_empty_file: erf
aliens.png: mime
largeiftrue.pcapng: pcapng
ch36_monitor.pcap: pcapng
webscraper.py: unknown
captype: "topsecret" is a directory (folder), not a file.

It’s easy to parse this format with awk. awk -F ': ', where $1 is the filename and $2 is the filetype. Any errors will put captype: in place of the filename.

When Your Pcap Extension != Filetype

You may have a file that has a .pcap extension but is actually a .pcapng file. This can easily happen if you save to a file like tshark -w example.pcap without specifying an encoding. tshark will default to pcapng, so you’ll have pcapng data with a pcap extension. While tshark and friends will read the encoding and not the extension, other programs may not be as forgiving.

Correcting Script

It’s easy to make this mistake as defaulting to pcap/pcapng varies by Wireshark utility. For example, if we save packets without explicitly setting the capture type using tshark’s -F, we’ll have a pcapng file with a pcap extension.

$ tshark -c 100 -w example.pcap
Capturing on 'Wi-Fi: en0'
100
$ captype example.pcap
example.pcap: pcapng

To automatically fix this problem, you can use this one-liner. If the filetype is different from the extension, the file is moved to the correct extension.

# If captype doesn't know which filetype a file is, it will classify it as "unknown"
# For any captype or awk error condition, mv's 2nd arg collapses to '' and mv will error.
mv -n $file "$(captype $file | awk -F ': ' '{ if ($2 != "unknown") print "'${file%.*}.'"$2}')"

Pcap Docs

Pcap Dissection

Libpcap Programming

Pcapng Docs

Pcapng Dissection

Pcapng Articles

  • Wikipedia: List of file signatures: How to know from the first few bytes “file magic” of a file what its type is.
  • 2016-01, Algis Salys, Pcap and Pcapng: pcap, pcapng, and converting between the two