Portland State University’s Security Club, the void* vikings, is proud to announce that we will be presenting at BSides PDX later this week.

We are presenting “A Hands on Guide to CTFs for the Uninitiated” in the Walkthroughs track at 11am. There will be a presentation about CTFs in general, strategies to get the most out of them, especially if you are a nontraditional player, heartwarming stories, advice, and challenge walk throughs selected by audience vote.


Note that the content has moved to a better home on github, https://github.com/anaaktge/cheribuild/wiki !

We hope to move into the main CheriBuild repository soon!

Below is the original content for posterity.


Hello! Below is my attempt to condense what a new contributor might need to know about CheriBSD, standing up applications on CheriBSD, and to some extent the broader CHERI (Capability Hardware Enhanced RISC Instructions) project, and its community into something reasonably digestible. It is based on my own experience, far from exhaustive, and at the rate things are moving, likely to be out of date before the metaphorical ink is dry.  If you are interested in the hardware aspect, please get in touch with the team.

Still, I hope it will help you understand a bit more about how the project operates and help you get oriented and comfortably settled into the team. We would like to have an expanded, diverse, and welcoming contributor community. One of the greatest strengths of this project is its people. This has been the most supportive and approachable OSS (Open Source Software) community I have ever worked with, and I hope your experience will mirror mine. 

I hope eventually to get things like this onto a wiki, with regular maintenance; for now something is better than nothing.

If you remember nothing else from this post: the researchers are very friendly and when in doubt reach out on slack, the mailing list, or github.  

Assumptions About You

    I assume that you’re already somewhat familiar with the CHERI project. If not, now might be a good time to at least skim through cheri-cpu.org. (For the enthusiastic reader, there are links to additional content later.)

I assume you are reasonably technical, interested in security, software, and/or computer architecture. I assume you largely know how to program and are at least familiar with the C language and pointers. Perhaps you are a motivated student in an undergraduate computer science program, or a software engineer looking for an interesting OSS project or considering work with the upcoming Arm Morello Board , or a graduate student looking for an interesting and novel research substrate.

    If you feel you are not quite at that level, it is certainly still possible for you to contribute.  Despite its decade of history, CHERI remains a young project and so has room for everything from documentation through small prototypes of novel system features through porting applications or even entire operating systems.  Please do not look down on or underestimate the value of noncode contributions. Tests and documentation are always in need of more attention and are often more valuable than new feature contributions over a project lifetime. Tutorials would also be extremely helpful at this juncture. 

Why contribute now?

The CHERI project began in 2010, and has been a small team of researchers, most of which are housed in a single building. Typically when onboarding a new contributor, someone either flies out to them or they travel to that building. 

That model isn’t terribly scalable, and with the announcement of real, physical CHERI hardware landing in 2021, the Arm Morello Board , there is a growing interest, and an increasing number of engineers and companies who want or need to contribute to the project. The UK government will also be funding UK universities and companies to perform research and build demonstrators/prototypes around CHERI as part of the Digital Security by Design initiative.

What are capabilities anyway? 

    Capabilities as a design concept are relatively old and relatively intuitive: something  should only be able to interact with resources for which it has (unforgeable) rights. Capabilities can be viewed as a way to implement the principles of least privilege and privilege separation.  They have been implemented in hardware, software, and a combination of the two, at least in a research setting.  Wikipedia’s Capability-based security article has a good list of capability supporting systems that managed to escape the lab. Traditionally, cost of re-development/re-implementation or performance has kept capability systems out of wider use. 

Do they replace FreeBSD Capsicum capabilities?

    They do not. Capsicum capabilities in FreeBSD (and by extension CheriBSD) is analogous to seccomp on Linux. It’s used at a per process level in user space to restrict system calls.  Nor are they related to the withdrawn POSIX capabilities.  

CHERI by contrast, is an architectural feature that extends existing architectures which can be used for things like low level memory safety. CHERI capabilities are most often used like fat pointers carrying additional benefits.  As a general rule, the closer you are tied to the hardware, ie compiler, language runtime, or operating system, the more you care about the CHERI features.

Ideally, a security conscious application would make use of both OS-based sandboxing and CHERI protections, although it may not be aware of the CHERI features it uses through a trusted computing base (like a compiler).   

Where are capabilities used in CheriBSD?

    CheriBSD is a CHERI-enabled adaptation of the FreeBSD operating system.  CheriBSD tries to stay up to date with FreeBSD plus or minus a few weeks. At the time of this writing, userland, the level at which normal applications and users operate, runs in pure capability mode, where old school, unprotected C style pointers are not allowed. The kernel itself however, is still not fully ported over to using capabilities within itself. There is ongoing effort on a pure capability kernel on a branch.  It’s one the many areas where we’d love some help. 

Is CheriBSD the same as CheriOS ?

    CheriBSD is not the same thing as cheriOS. CheriBSD is a CHERI-fied version of FreeBSD, which was designed long before CHERI, and whose internals may not all be able to benefit from all of CHERI’s features. For example, employing CHERI memory safety seems to work well with the UNIX design and implementation, but using CHERI-based compartmentalization requires greater rethinking of OS software abstractions.  In contrast, CheriOS is a new microkernel built from scratch with CHERI capability hardware in mind. I have not worked with it, however it’s a very intriguing design space. If that interests you, please see the CheriOS github page and reach out on the dedicated channel on the CHERI CPU slack.

Where can I contribute to CheriBSD?

    I have a few suggestions; however, don’t feel limited by them. The team is bursting with ideas and they’re open to hearing more.

Some of the eternal suggestions include creating and improving documentation, and expanding test coverage. If you feel your C language mastery is weak, test writing might be a really great way to build that skill up and contribute at the same time.  

    Converting popular utilities, libraries, and even full-scale applications to use capabilities would certainly be handy. The first step is to ensure that they work well with CHERI C/C++, which is often straightforward and consists of helping older code better differentiate its integer types from capability types. There are some subtleties, and the CHERI developers will be happy to help out. There is a technical report documenting CHERI C/C++

Sounds awesome! How do I communicate with the community?

    I’m so glad you asked! Please join us on the mailing list, slack, and github. The mailing list can be found at https://lists.cam.ac.uk/mailman/listinfo/cl-cheri-discuss and if you are not from Cambridge, please ignore the scary red box and treat it as a normal mailman list. The slack is at cheri-cpu.slack.com and please feel free to explore the channel list. The CHERI main github page is a bit overwhelming with 120+ repositories. The two you’ll probably want to start with are the CheriBSD github and Cheribuild github.

Where can I find out more?

    In the immortal words of Lego Batman, “prepare yourself for some….reading”! The whole list of publications can be found on CHERI Publications page. I don’t recommend you start there. Academic papers represent truth at time of publication, and may no longer be current. 

I’ve taken to renaming any CHERI related pdf I download, because the names are not descriptive for end users.  

I recommend you go in the approximate following order, using the Architecture document as a reference as you read the other things.

What tools would be good to know beforehand?

  • QEMU

    QEMU is a processor emulator, sometimes classified as a userland hypervisor. Whichever way you slice it, there is a forked version that supports CHERI-MIPS, CHERI-RISC-V, and Morello. The CHERI-RISC-V is the principal focus for development, but Qemu CHERI-MIPS was the stable option until earlier this year.

Note that I was using MIPS, however new users should now start with RISCV invocations. I have both listed where applicable.

  • FreeBSD Basics

    CheriBSD is based on FreeBSD. Much of the team has extensive FreeBSD experience. I don’t however, and definitely wandered in a few weeds as a result. If you would like to learn more about FreeBSD and how the kernel is structured, there is the FreeBSD Handbook. If you are more of a textbook learner, or would like a more polished survey of the kernel than the Handbook, The Design and Implementation of the FreeBSD Operating System 2nd ed (ACM linkamz link ) was recommended to me by a long time FreeBSD committer.  Yes, one of the authors is part of CHERI and worked on Capsicum.

  • C Language 

    CheriBSD is primarily written in C and the principal CHERI targets are TCBs (Trusted Computing Bases), which are generally written in C or in something that secretly uses C. As a result, you’ll probably have a more pleasant time if you are conversant with C, and pointers in particular. I tend to have my trusty K&R C book within reach whenever I revisit C. You may want your C reference of choice handy as you read the CHERI C/C++ Programming Guide.  I understand that C is no longer taught as often and C fluency is much less common now, so if this a weak area for you, please be patient with yourself. 

    If you’re in the mood for a spot of UNIX & C history, consider reading The UNIX time-sharing system, by Dennis M. Ritchie & Ken Thompson (1974) . It may help you understand FreeBSD and C a little better. 

  • Command Line Interface (CLI)

    Cheribuild, qemu, and various other bits largely live and run out of your terminal. The build & install process largely assumes familiarity with the CLI. Familiarity with your package manager might also save you some pain and troubleshooting later on. 

  • Git & Github

As I mentioned earlier, the CHERI main github page is a bit overwhelming. However, the documentation on the build system is very good, Alex has done a stellar job, and can be found on Cheribuild github. If you are interested in CheriBSD, then of course you should also check out the CheriBSD github.

How do I set this up? Starting with Cheribuild

    Cheribuild is a nontrivial feat of engineering in its own right and it’ll be your best friend for getting started. The documentation is excellent and kept up to date. Alex works very hard on it and it shows. The overall set up is to install some requirements, set up the directory structure, clone a repo, and let Cheribuild work perhaps while you do some light CHERIreading. Remember that ‘-f’ is your friend if you tend to wander away from your terminal during installations. 

    My systems are Ubuntu, I understand this works on FreeBSD and OSX, however I have not tried it and cannot speak to the installation process.

    Be aware that when you re-start Qemu and rebuild, it will prompt you, but likely overwrite the CheriBSD disk image and all the changes you might have made to it. That’s intentional, but may not suit your development style. Pay careful attention to the Cheribuild documentation!  

    I did not know about it when I was setting up, but there is a –force option to automatically suppress the y/n prompts. I recommend you try it out for installation.  If you run into issues, please file a bug. We’d appreciate it. 

Output from my terminal: 

  • a@a:~$ sudo apt-get install libtool pkg-config clang bison cmake ninja-build samba flex texinfo libarchive-dev libglib2.0-dev libpixman-1-dev  libbz2-dev  libarchive-tools
  • a@a:~$ mkdir ~/cheri
  • a@a:~$ git clone https://github.com/CTSRD-CHERI/cheribuild ~/cheri/cheribuild

Can I set up it on Windows?

    Well, no not really.  Your best bet is to run the Qemu setup in a VM. The team thinks Cygwin or WSL might work, maybe, but they have never tried it and no one has any plans to either.  Perhaps a place to contribute if you are a Windows programmer?

How do I start/stop Qemu CheriBSD?

risv (what you should use):

a@a:~$ ~/cheri/cheribuild/cheribuild.py qemu llvm cheribsd-riscv64-purecap disk-image-riscv64-purecap run-riscv64-purecap

mips (what I used):

a@a:~$ ~/cheri/cheribuild/cheribuild.py qemu llvm cheribsd-mips64-purecap disk-image-mips64-purecap run-mips64-purecap

(Assuming you have installed Cheribuild from the above section, this will build the development environment and then start the mips version, not the RISC-V one)

Default account is ‘root’ with no password. I’m told this is a tradition from the FreeBSD live boot CD, although I’ve never seen one of these mythical things in person. 😀 I generally run ‘uname -mrs’ after starting CheriBSD for a sanity check, though that’s just me.   

By the way, the cleanest way to exit the whole thing is to run ‘shutdown -p now’ at the CheriBSD terminal prompt. Be advised that it may take a surprisingly long amount of time to shutdown and exit. That’s normal. 

If you don’t care about ruining the disk image or keeping your install stable, i.e. you’re going to rebuild it anyway, “Ctrl-A x” will kill QEMU. I generally don’t do this, however most of the team does. 

I’ve got it running!

Yay! Congratulations! Do a happy dance, then poke around a bit and explore! 

How do I test stuff in CheriBSD ?

    Cheribuild to the rescue! There’s a ‘–test’ argument to run the existing tests. It’s not as extensive as it could be, but it does exist. For new work, there is still a lot of manual testing by the developer and the reviewer. At this time, there is no formal QA verification, although Alex has been working hard to improve the CI (Continuous Integration) and run Jenkins on master.  

Your test invocation is:    

a@a:~$ ~/cheri/cheribuild/cheribuild.py qemu llvm cheribsd-riscv64-purecap disk-image-riscv64-purecap run-riscv64-purecap --test

My test invocation was:

    a@a:~$ ~/cheri/cheribuild/cheribuild.py qemu llvm cheribsd-mips64-purecap disk-image-mips64-purecap run-mips64-purecap --test

    New stuff lands very frequently, so you may find a clean master breaks. Don’t panic. We’re getting better, and most of the main repositories stage changes through a “dev” branch, though it still happens. If you run into issues with new changes, Speak up! Please!Related to testing, Alfredo has syzkaller, a kernel fuzzer, running regularly.  The ISA(Instruction Set Architecture) and other parts have testing and verification procedures of their own, however my focus is primarily on CheriBSD.

How do I debug stuff in CheriBSD?

There are 3 flavors of GDB to consider: inside-inside, inside-outside, outside-outside. Inside-inside GDB is cross-compiled, hybrid aware, running in CheriBSD to debug CHERI userspace. inside-outside GDB is a CHERI aware host executable used to debug the CheriBSD kernel with a debug server, such as the one provided by QEMU. Outside-outside GDB is a host process debugging a host executable like QEMU itself. Outside-outside is the least used and most likely not one you’ll reach for often. For the CHERI cases, you’ll want to check out the latest debugging over at Building GDB

For working in the CheriBSD kernel, remote gdb debugging is the way to go. You’ll be looking for the extra flag –wait-for-debugger, wait on cheribuild, then follow the directions it prints for the type of debugging you’d like to do. –wait-for-debugger is most useful for kernel debugging rather than userland debugging.

For me, this tends to be fragile, and you’ll need to be _very_ close attention to the directions that cheribuild spits out for attaching the debugger to the kernel or userland. Very, very close attention. I personally use two terminals side by side when setting it up.  

For working *on* CheriBSD, in userspace, gdb is present, but not built by default. You’ll need to add an extra flag to the starting incantation. Please check the Cheribuild documentation for the latest details. Ultimately you’ll be interested in running gdb-run.sh, which will start a program in GDB and print a backtrace once it stops, which can save a lot of your sanity.

Your (kernel) debugger invocation is: 

a@a:~$ ./cheri/cheribuild/cheribuild.py qemu llvm cheribsd-riscv64-purecap disk-image-riscv64-purecap run-riscv64-purecap gdb-native --wait-for-debugger

My (kernel) debugger invocation was: 

a@a:~$ ./cheri/cheribuild/cheribuild.py qemu llvm cheribsd-mips64-purecap disk-image-mips64-purecap run-mips64-purecap gdb-native --wait-for-debugger

How do I find a Good First BugTM ?

    I’m afraid that is a luxury that this project does not yet have. However, what we lack in organization, we make up for in engagement. Ask in slack or on the mailing list with your interest and skills, and we’ll try to match you as best we can. 

    If you were interested in compiling and maintaining such a list as a project, that would be a nice gift to future contributors. 

How do I find a project or task generally on CHERI?

    As of this writing, it’s the same friendly process as above.  On the bright side, there is a high level of interpersonal interaction. There’s a good chance the whole team will know your name by the time your contribution lands.

How do I find a Mentor?

    We don’t have a formal mentorship program at this stage (also a luxury of bigger projects). That doesn’t mean folks aren’t interested in helping new contributors, yet it does mean that you’ll need to be a bit more proactive about asking for help. If it helps you, you can always ask someone to mentor you explicitly. I recommend slack over the mailing list for seeking a mentor.

What are the development style requirements?

    The style guide for CheriBSD is the venerable, cryptic style(9), as it is for FreeBSD. There is a top level clang format file in the source tree and there exist plugins for Vim & Emacs.  If you prefer VScode, as far as I can tell you’re out of luck at this time. 

    The other style note worth mentioning, is that since CHERI is an acronym, it is usually capitalized. CheriBSD and Cheribuild are amongst the most prominent exceptions. 

How do I ask for a code review?

    There is no formal process for asking for review or who should be assigned a given review. Ideally, whoever you’re working on changes with will be able to advise you or do the review. The github suggestions are generally good, and those folks can redirect you if the algorithm makes an incorrect recommendation. If all else fails, please ask in slack and/or make a Pull Request(PR). Mark it with ‘work in process’ and with a description as informative as possible to help things move smoothly. 

I think I found an issue, but I’m not sure. What should I do?

    If you think you found something amiss, you probably did. Please let us know either by filing an issue on the appropriate github repo, or asking in slack if you aren’t sure which repo it belongs in. Thanks!

    If you found something sensitive or non public? Please reach out on slack instead of github. If it’s a flaw in an upstream source, that’s a bit more sensitive and should probably be treated as nonpublic at first to be safe.  If you found something like a security flaw in FreeBSD, please reach out directly on slack to Brooks Davis.

Is there a Code of Conduct?

    The short answer is ‘not yet’. The team is working towards a proper CoC. It’s not an overnight process. The CheriBSD group will look over the new FreeBSD one, and if it checks out, may decide to adopt it. 

How are decisions made? What is the governance model ?

    There is no formal or official governance model.  Originating as a small research project, decisions were made by the research team. There are now numerous private contracts between various organizations that are involved, and those details are nonpublic. I understand that currently much is still sorted in closed meetings and then disseminated around. 

It’s not clear at this time how much influence a volunteer contributor outside that structure can have, or what recognition is possible. It is a question the community will need to resolve as it grows and expands beyond labs and corporate research groups.


    I would like to thank the CHERI team for their patience and support.  In particular, Dr. Robert Watson, Dr. Wes Filardo, & Dr. Alex Richardson deserve extra thanks and credit for their continued feedback.

I have the honor of being the least interesting person on the panel for What is Lacking in Linux Security and What Are or Should We be Doing about This at the Linux Security Summit North America 2020. It will be on Wednesday, July 1st, 2:55pm – 3:55pm CDT.

Should be a fun time. I’ll also be lurking in LSSNA slack for the duration of the conference, eyeing everyone’s cute pets and witty oneliners enviously. 🙂

The Naked Trill project will be presenting our research in reverse engineering, corrupting, & fixing translators in Seagate firmware at bsidesPDX in a few weeks.

We’re in track A, 1pm on Friday, October 25th. Come say hi 🙂

There are number of excellent talks and workshops planned that are way cooler than I am. Check out the full schedule

Introducing the QultoftheQuantumQapybaras

QultoftheQuantumQapybaras is a newly formed all female CTF team in the Portland, Oregon area. We are a sibling team of the older QultoftheQuantumQow. Both teams are based out of the nonprofit hackerspace PASCAL. To the best of our collective knowledge, we’re currently the only all female CTF team in the area. If there are others, please let us know.; we’d love some company.

DC Quals was the team’s first contest, and the first CTF ever for the majority of the team. There were six of us playing this round and we finished #119 out of 1262 teams, putting us in the top ten percent. I’m very proud of how well the team worked together.


Challenge prompt: “You know, we had this up and everything. Prepped nice HTML5, started deploying on a military-grade-secrets.dev subdomain, got the certificate, the whole shabang. Boss-man got moody and wanted another name, we set up the new names and all. Finally he got scared and unplugged the server. Can you believe it? Unplugged. Like that can keep it secret…”

It also came with a HINT file, containing the following string: ”Hint: these are HTTPS sites. Who is publicly and transparently logging the info you need? Just in case: all info is freely accessible, no subscriptions are necessary. The names cannot really be guessed. “

The Journey to the Flag

This write up is a combination of our collective notes during the solve. My teammates deserve at least as much credit as I do, especially as they took much better notes. I have elected not to edit my own failures out of the story, in the hopes of inspiring other CTFers not to be discouraged by theirs. The process is messy and that’s okay.

I initially ignored this challenge because someone else was working on it and settled on the RETURN_TO_SHELLQL challenge. At one point, a teammate asked me about certificate authorities because I used to work on Firefox. My curiosity was piqued. OK, I was stuck on shellql and needed a break. Same thing, right?

If the focus on certificates is confusing to you, please recall that the challenge mentions a certificate directly. The hint strongly links that to the certificates used by SSL/TLS for HTTPS traffic.  In this case we’re looking for a certificate issued to a url containing “military-grade-secrets.dev” subdomain.

CAs didn’t really show up in my corner of the Mozilla world unless there was drama around an organization’s application to be a CA. I did dimly remember that there was supposed to be some sort of oversight and audit trail, but not how it worked or if it still existed. As with many things, Certificate Authorities didn’t quite turn out the way the architects intended.

The hint brightened those dim recollections a bit, and I did some googling on ‘certificate authority transparency’, of which the most salient was this wikipedia article on Certificate Transparency . That closely matched my dim recollections about oversight and confirmed that the records should be publicly available. So, my foolproof plan was to find the certificate record, link it to a url, drop the url in the Wayback machine, retrieve the secret page, and find the flag.

This is where most of the wandering in the weeds happened for me. I searched around for DNS, cert records, and found a couple sites like SSLChecker, MX tools, etc . I plugged in variants of “military-grade-secrets.dev” and ooo into several of these without success. RAWR! I _know_ there are historic records! I’ve been digging for them the whole time! After about 3 minutes of seeing red, I calmed down enough to connect the dots. The records I was looking at must only be current records. A quick google for ‘archived dns’ turned up this DNS archive.

The results out of that one were more promising. A search for our key string yielded “Military-grade-secrets.dev”, “www.military-grade-secrets.dev”, “secret-storage.military-grade-secrets.dev”, “Now.under.even-more-militarygrade.pw.military-grade-secrets.dev”. None of the links worked as is. So I proceeded to the next phase of my plan: plugging them all into the Internet Archive’s Wayback Machine.  I came up entirely empty handed for all of them, thought I had failed, and started looking for another angle.

My teammate overheard my Wayback attempts, and unlike me, can use tools correctly. 🙂  The Wayback Machine turned up exactly one record for “Now.under.even-more-militarygrade.pw.military-grade-secrets.dev”. , dated May 11th. It was a redirection to https://forget-me-not.even-more-militarygrade.pw/.  

That page was down of course, the server is supposed to be unplugged after all. Of course, the Wayback Machine had a nice copy which displayed the flag on the lower half of the page in a nice large font: OOO{DAMNATIO_MEMORIAE} .  

Thus, QultoftheQuatumQapybaras’ first flag was captured.

Hacker Quapybara by http://danigrillo.com/

Artwork by Illustrator Dani Grillo


    Please note that I did not name the project. It is a play on our respective surnames.

For several months, my research partner Mr.De4d and I have been spelunking in the depths of the Seagate firmware, whose internals are legendarily undocumented.

We have been studying a form of hard drive failure which regularly occurs in the wild in the Seagate Moose and Pharoah drive families. A seemingly undamaged drive powers on, spins up, gets stuck in busy, and is never recognized by the connected operating system. Mr.De4d has fixed several of these manually in the wild by forcing a translator rebuild. So we strongly suspect a corrupted translator in the firmware to be at fault.

Our objective is to build a tool or at least a procedure to programmatically fix the translator and make that available. The first step is to make ourselves a controlled corrupted translator to build and test code against. That is proving to be harder than anticipated. Work is ongoing at the time of writing.

Along the way, we have found a number of interesting things about the lists used by the Seagate translator. I had thought to wait until after we had finished our entire project to publish anything. However, we have received feedback that the data recovery community would really like to see what we have found so far even if it is unfinished.

A Bit of Background on Lists used in Firmware Translators

Conventionally, HDDs handle sectors going bad by building a translator that converts Logical Block Address(LBA) to a physical location, usually represented by the combination of cylinder, head, sector (CHS). Some sectors, part of the physical address, are defective during manufacture, others fail over time. The translator is responsible for taking LBAs and mapping them onto good sectors, skipping the bad ones. Conventionally it uses two-three lists to manage the types of bad sectors. Translators are expected to be rebuilt periodically to factor in changes to those lists. For a more thorough explanation of the translator and remapping, please see MJM’s bad sector remapping article.

They are:

  • G-List – Grown Defect List, list of defects accumulated over time, expected to dynamically grow over time
  • P-List – Primary Defect List, list of defects found at factory at start of device’s lifetime, expected to be static through the life of the device
  • T-List – Thought to stand for Tracks Defects List, although there are other theories. Generally believed to be dynamic.

What We Found in the Firmware

We were expecting to find three, maybe four lists when we popped open the terminal and starting poking around. We were to looking to identify and overfill one of the Grown Defects Lists to simulate the broken translator found in the wild. So we have found eight lists, and we are not certain that we have discovered them all yet.

Seagate NameTypeNotes
Alt ListG– Alt List: entries  by Logical Block Address(LBA) or sector.
– Synced with G-List.
– Changes over time as defects accumulate. Expected to be dynamic.
G-ListG– G-List: entries by sector only, unlike Alt List
– Synced with Alt List
– Changes over time as defects accumulate. Expected to be dynamic.
LBA List?– Might be an alias for the Alt List
Non Resident G-list (NRG)P– The 2nd primary list generated at the factory after the P-List is generated.
– Generated after a 2nd Self-Scan Test
– Unclear why. Expected to be static.
P-listP– “Classic” P-List; Factory defect list used to generate the translator. Expected to be static
– Generated after initial Self-Scan Test performed at factory level.
T-List?– Cylinders are referenced when contents displayed and it looks similar to the Alt List
– May be a copy of the Alt List or may be a true T-list (track defect list).
User Slip Defect List (USDL)?– May be related to G-list, entries the same, but each entry has additional information
System Slip Defect List (SSDL)?– Able to view after ‘checks T-list” cmd(s) are run, suggesting a relationship to the T-List
– Much, much smaller than the USDL

Bottom Line:

       This one does what it says on the tin in a clear, readable format that should be accessible to most of the population. Those with a technical background will need to skim large sections, thankfully the book’s clear format makes that easy.  

  1. Did I learn something?
    1. I developed a high level understanding of the digital forensics field.
    2. I learned quite a few interesting tidbits about Windows.
    3. I learned quite a bit about the legal tangles involved in digital evidence.
  2. Did I enjoy the time spent reading it?
    1. Yes. I really enjoyed the anti-forensics chapter.
  3. Would recommend to:
    1. Those studying law, studying criminal justice, or police officers in field work.  Chapter 7 on legal aspects and Chapter 10 on mobile devices would be particularly relevant.
    2. Those considering a career in forensics. The author provides many details about the ‘daily grind’ and some professional pitfalls of being an examiner throughout the book.
    3. Career changers who want to get into tech but aren’t sure that full on software development suits them and are looking for related career paths.
    4. Veteran lawyers struggling with digital evidence reports or are new to digital evidence in general. Chapter 10 on mobile device is especially relevant, although Chapter 7 is unlikely to contain anything they don’t already know.
  4. Would not recommend to:
    1. DevSecOps and Infosec folks looking to better their intrusion postmortems and guidance on how to conduct their own detection & collection, effectively forensics, during or after an attack.
    2. The technically inclined looking for a deeper insight into the software that powers digital forensics. This was very much focused on the squishier side of the keyboard.

On Content:

      In general, I found the content to be good quality and easily digestible. The presentation of the forensics process, the factors that influence its application in the digital space, and what makes for a successful investigation from start to finish stand out in particular.

      The preface of the book discusses the intended audience and the expectation that readers have a ‘fundamental understanding’ of computers. The explanations for different bits of technology were so vague and so high level that they must be meant for those with no understanding of computers, not a ‘fundamental’ one.  I found the partial sector data recovery and the encryption examples to be especially drawn out and hand wavy, and advise anyone technically inclined to skip those sections.

      “You cannot trust closed-source crypto”  on page 87 is probably the boldest line in the entire book. The author is careful to present balanced arguments around open source versus closed source forensic software suites, the possession of disk wiping tools, etc. So it was a bit startling to see him take an unequivocal stance around something that is still controversial in public discourse.

      Chapter 6 on anti-forensics is most likely to appeal to those in tech with a pen-ten or tinkering bent, and it was my favorite chapter. I was surprised to learn that Steganography is still one of the most effective ways to foil forensics; even if the examiners know to look for it in the first place. I remember learning about it while researching Benedict Arnold in high school and doing an in class demo of the spy’s technique. I had thought it largely obsolete and I was wrong.

      I had some smaller gripes with the book. I didn’t love the windows exclusive focus, although I understand the stated rationale. Still, I expected some mention of common software found on servers and networking equipment in Chapter 9 on network forensics. I also thought the treatment of IRC was unbalanced. IRC is used by many developers working at tech companies, universities, and on open source projects, but only its use by criminal elements was noted. None of its legitimate uses were mentioned.

      I also wish the author had discussed the economics of the field more directly. He mentions various costs when discussing trade-offs for accreditation, for multiple tooling suites, and for physical supplies. He never directly discusses income. Reading between the lines around the emphasis on court admissibility that reoccurs throughout the book, I infer that the justice department and lawyers are the main clients of digital forensics. I think a direct discussion of that and its impact on the field would have been very useful.  

      The author repeats warnings and concerns about the gap between the speed of technology and court-admissible forensics tools throughout the book but avoids discussion of how to fix the problem. It’s not possible to discuss how to fix that widening gap without discussing the the influence and responsibility of the economic engine behind the field.

On Presentation:

      Writing was clear, concise and easy to follow. The chapter arrangement and section arrangement within each chapter flows smoothly. My only presentation nit is that the font size for the interviews in the book is much smaller than the paragraph text; it lost readability.

Searching for stuff, either in your own history or on the web, is an integral part of the browsing process.  The UI for searching is important to a great user experience. For us, that UI is part of what we call the Awesomescreen. Firefox Desktop has an Awesome bar, we have an Awesomescreen. Look for these changes in Firefox 44.

New Search Suggestions from Your Search History

Previously, if you opt into search suggestions, we only showed you suggestions from your default search engine (if your default search engine supports it). Now we can also show you things you’ve searched for in the past on your mobile device.  These suggestions from your search history are denoted by a clock icon. Anthony Lam designed the shiny new UI you’ll see below.

We’ve also updated the layout and styling of the search suggestion ‘buttons’ to help you spot the one you want faster and make the whole process easier on your eyes. 😉

We believe in user choice and control, so this is controlled by a separate setting.  If you don’t like, you can just turn it off without losing suggestions from your preferred search engine. Simply go to Settings > Customize > Search > “Show search history” and uncheck the checkbox. Likewise, if you dislike suggestions from search engines, you can disable that without losing suggestions from your own history. For that, go to Settings > Customize > Search > “Show search suggestions” and uncheck that box.


New saved searches on phone



New saved search suggestions on tablet

Updated Permissions Prompt

We’ve update the prompt to be feel more modern and integrated with the browser. You may have noticed that the doorhanger prompts recently got a makeover, courtesy of my colleague Chenxia.


New permissions prompt for opting into search suggestions

Engineer goes off and adjusts the plan. At a high level,it looks like this:

    Client side code

        Set up

            snippet lets users know there is a new options in preferences

            if the app locale does map to a supported language, the pref panel is greyed out with a message that their locale is not supported by the EU service

            if not, user clicks ToS, privacy policy checkbox, confirms language

            app contacts server

            if server not available, throw up offline error

            if server available, upload device id, language, url list

            server sends back the guid assigned to this device id

            notify user setup is complete

            enable  upload service

        When new tab is opened or refreshed

            send msg to server with guid + url

        Turning off feature

            prompt user ‘are you sure’ & confirm

            notify server of deletion

            delete local translated pages

Server side code

    Set up

        poked by client,

        generate guid

        insert into high risk table: guid+device id

adds rows for tabs list (med table)

adds rows for the urls (low table)

    Periodic background translation job:

        finds urls of rows where the translated blob is missing

        for each url, submits untranslated blob to EU’s service

        sticks the resulting translated text blob back into the table

    Periodic background deletion jobs:

        finds rows older than 2 days and evicts them in low risk table & medium risk tables

        find rows in sensitive table older than 90 days and evict.

secure destruction used.

        user triggered deletion

            delete from sensitive table. secure destruction

            delete from medium table

    Database layout

        sensitive data/high risk table columns: user guid, device id

            maps guid to device id

        medium risk table columns: user guid, url

            maps guid to tabs list

        low risk table columns: url, timestamp, language, blob of translated text

            maps urls to translated text

The 4th Meeting

Engineer: Hey, so what do you guys think of the new plan? The feedback on the mailing list was pretty positive. People seem pretty excited about the upcoming feature.

Engineering Manager: indeed.

DBA: much better.

Operations Engineer: I agree. I’ll see about getting you a server in stage to start testing the new plan on.

Engineer: cool, thanks!

Engineer: Privacy Rep, one of the questions that came up on the mailing list was about research access to the data. Some phd students at the Sorbonne want to study the language data.

Privacy Rep: Did they say which bits of data they might be interested in?

Engineer: the most popular pages and languages they were translated into. I think it would really be just the low risk table to start.

Privacy Rep: I think that’d be fine, there’s no personal data in that table. Make we send them the basic disclosure & good behavior form.

Engineering Manager: A question also came to me about exporting data. I don’t think we have anything like that right now.

Engineer: No, we don’t.

Privacy Rep: well, can we slate that do after we get the 1.0 landed?

Engineering Manager: sounds like a good thing to work on while it’s baking on the alpha-beta channels.

Who brought up user data safety & privacy concerns in this conversation?

Engineer, Engineering Manager, & Privacy Rep.