From Collaborative RCE Knowledge Library

Jump to: navigation, search

OS Independent Internals

Item name: Introductory Intel x86: Architecture, Assembly, Applications, & Alliteration
Rating: 5.0 (1 vote)
Author: Xeno Kovah                        
Home URL:
Last updated: June 27, 2011
Version (if appl.):
Direct D/L link:
Description: This is a 2 day class which is freely available to watch. You can also take the materials and use them to teach your own classes.


Intel processors have been a major force in personal computing for more than 30 years. An understanding of low level computing mechanisms used in Intel chips as taught in this course serves as a foundation upon which to better understand other hardware, as well as many technical specialties such as reverse engineering, compiler design, operating system design, code optimization, and vulnerability exploitation.

25% of the time will be spent bootstrapping knowledge of fully OS-independent aspects of Intel architecture. 50% will be spent learning Windows tools and analysis of simple programs. The final 25% of time will be spent learning Linux tools for analysis.

This class serves as a foundation for the follow on Intermediate level x86 class. It teaches the basic concepts and describes the hardware that assembly code deals with. It also goes over many of the most common assembly instructions. Although x86 has hundreds of special purpose instructions, students will be shown it is possible to read most programs by knowing only around 20-30 instructions and their variations.

The instructor-led lab work will include:

* Stepping through a small program and watching the changes to the stack at each instruction (push, pop, call, ret (return), mov)
* Stepping through a slightly more complicated program (adds lea(load effective address), add, sub)
* Understanding the correspondence between C and assembly control transfer mechanisms (e.g. goto in C == jmp in ams)
* Understanding conditional control flow and how loops are translated from C to asm(conditional jumps, jge(jump greater than or equal), jle(jump less than or equal), ja(jump above), cmp (compare), test, etc)
* Boolean logic (and, or, xor, not)
* Logical and Arithmetic bit shift instructions and the cases where each would be used (shl (logical shift left), shr (logical shift right), sal (arithmetic shift left), sar(arithmetic shift right))
* Signed and unsigned multiplication and division
* Special one instruction loops and how C functions like memset or memcpy can be implemented in one instruction plus setup (rep stos (repeat store to string), rep mov (repeat mov)
* Misc instructions like leave and nop (no operation)
* Running examples in the Visual Studio debugger on Windows and the Gnu Debugger (GDB) on Linux
* The famous "binary bomb" lab from the Carnegie Mellon University computer architecture class, which requires the student to do basic reverse engineering to progress through the different phases of the bomb giving the correct input to avoid it “blowing up”. This will be an independent activity.

Knowledge of this material is a prerequisite for future classes such as Intermediate x86, Rootkits, Exploits, and Introduction to Reverse Engineering (all offered at
Also listed in: Generic Malware Analysis Articles, Generic Reversing Technique Tutorials, X86 Internals Tutorials
More details: Click here for more details, images, related URLs & comments for this item! (or to update its entry)

Item name: Intermediate Intel x86: Architecture, Assembly, Applications, & Alliteration
Rating: 0.0 (0 votes)
Author: Xeno Kovah                        
Home URL:
Last updated: July 15, 2011
Version (if appl.):
Direct D/L link:
Description: This is a 2 day class which is freely available to watch. You can also take the materials and use them to teach your own classes.


Building upon the Introductory Intel x86 class, this class goes into more depth on topics already learned, and introduces more advanced topics that dive deeper into how Intel-based systems work.

Topics include, but are not limited to:

•Physical and virtual memory and how a limited amount of physical memory is represented as much more virtual memory through a multilevel paging system. We will also talk about memory segmentation.
•The hardware basis for kernel versus userspace separation and how software transitions between the two. This portion answers the question of why does x86 have 4 “rings”, with ring 0 being the most privileged, and ring 3 being the least.
•Hardware and software interrupts, and how they are the basis for debugging.
•Input/Output instructions and how these allow the CPU to talk to peripherals.

Example applications include showing how hardware and memory mechanisms are used for software exploits, anti-debug techniques, rootkit hiding, and direct hardware access for keystroke logging.

This material includes labs on:
•Using WinDbg to perform kernel debugging on a virtual machine (which is equally applicable for debugging a real machine.)
•Using a custom WinDbg plugin to examine the Local (memory segment) Descriptor Table (LDT), and Global (memory segment) Descriptor Table (GDT) in order to understand how Windows sets memory segment ranges and permissions for userspace and kernel space.
•Using WinDbg and the !pte command to understand how Windows organizes its paging structures which map physical memory to virtual memory.
•Investigating where exactly the XD/NX bit is set in order to make memory as non-executable (which Microsoft calls Data Execution Prevention (DEP)), to prevent some types of exploits from succeeding.
•Using the Read Timestamp Counter (RDTSC) instruction to profile code execution time. Also, using a profile of code execution time to change a program’s behavior in the presence of a debugger (e.g. executing different code if the code appears to have been stopped at a breakpoint.)
•Printing information about task state segments, which hold information that is used to find the kernel stack when an interrupt occurs.
•Watching what does and doesn’t change when a software interrupt is used to transfer control from userspace to kernel.
•Reading the Interrupt Descriptor Table (IDT) and understanding the security implications of changes to it.
•Understanding how RedPill uses the IDT in order to detect that a system is virtualized.
•Having a process read its own memory when a software breakpoint is set, in order to see how a debugger will change memory to set the breakpoint but hide the change from the user.
•Watch how hardware-based breakpoints manipulate dedicated debug registers.
•Using port input/output to access the backdoor communications channel that VMWare uses in order to send copy/paste, mouse movement, and other events in and out of a VM.
•Using port I/O in order to talk directly to the PS2 keyboard controller in order to sniff keystrokes or flash keyboard LEDs.

Knowledge of this material is strongly encouraged for future classes such as Rootkits. (offered at
Also listed in: Generic Malware Analysis Tutorials, Generic Reversing Technique Tutorials, Windows Internals Tutorials, Windows Malware Analysis Tutorials, X86 Internals Tutorials
More details: Click here for more details, images, related URLs & comments for this item! (or to update its entry)

Item name: Pinczakko's guide to Award BIOS reverse engineering
Rating: 0.0 (0 votes)
Author: Pinczakko                        
Home URL:
Last updated: 2010
Version (if appl.):
Direct D/L link:
Description: 1. Foreword
2. Prerequisite
2.1. PCI BUS
2.2. ISA BUS
3. Some Hardware Peculiarities
3.1. BIOS Chip Addressing
3.2. Obscure Hardware Port
3.3. "Relocatable" Hardware Port
3.4. Expansion ROM Handling
4. Some Software Peculiarities
4.1. Call Instruction Peculiarity
4.2. Retn Instruction Peculiarity
5. Our Tools of Trade
5.1. What do we need anyway?
5.2. Intro to IDA Pro Techniques
5.2.1. Introducing IDA Pro
5.2.2. IDA Pro Scripting and Key Bindings
6. Award BIOS File Structure
6.1. The Compressed Components
6.2. The Pure Binary Components
6.3. The Memory Map In The Real System (Mainboard)
7. Disassembling the BIOS
7.1. Bootblock
7.1.1. "Virtual Shutdown" routine
7.1.2. Chipset_Reg_Early_Init routine
7.1.3. Init_Interrupt_n_PwrMgmt routine
7.1.4. Call To "Early Silicon Support" Routine
7.1.5. Bootblock Is Copied And Executed In RAM
7.1.6. Call to bios decompression routine and the jump into decompressed system bios Enable FFF80000h-FFFDFFFFh decoding Copy lower 128KB of BIOS code from ROM chip into RAM Disable FFF8_0000h-FFFD_FFFFh decoding Verify checksum of the whole compressed BIOS image Look for the decompression engine Decompress the compressed BIOS components The format of the LZH level-1 compressed bios components The location of various checksums The key parts of the decompression routine Shadow the BIOS code Enable the microprocessor cache then jump into the decompressed system BIOS
7.2. System BIOS a.k.a Original.tmp
7.2.1. Entry point from "Bootblock in RAM"
7.2.2. The awardext.rom and Extension BIOS Components (lower 128KB bios-code) Relocation Routine
7.2.3. Call to the POST routine a.k.a "POST jump table execution"
7.2.4. The "segment vector" Routines
7.2.5. "chksum_ROM" Procedure
7.2.6. Original.tmp Decompression Routine for The "Extension_BIOS Components"
7.2.7. Microcode Update Routine
8. Rants and Raves
9. Closing
Also listed in: X86 Internals Tidbits
More details: Click here for more details, images, related URLs & comments for this item! (or to update its entry)

Item name: Super-secret debug capabilities of AMD processors !
Rating: 0.0 (0 votes)
Author: Czernobyl aka Czerno                        
Home URL:
Last updated: June 12, 2014
Version (if appl.): 1.0
Direct D/L link: N/A
Description: Secret debugging extensions in AMD K7 processors
Here unveiled by Czerno - Mail : <me AT>
Original article : December, 2010. This revision : June, 2014.
Reason for revision : contents made more accurate, shorter and hopefully, clearer.

Copyleft (c) Czerno. Please keep attribution where it belongs.

The author shall not be held responsible for any errors or inaccuracies, blah-blah...

Click the "more details" button or link downpage to view additional notes!

Very important : you can help! Yes, YOU!

- By doing your own trial of the features and contacting us over any errors/inaccuracies/complements you find!
We want to assert, in particular, whether the features we found in Athlon-XP are present, possibly modified, in the newer generations of AMD CPUs.
- By updating debuggers, plugins and toolz so they can make full use of the new features.


_Summary_  :

AMD K7 (Athlon-XP, etc.) processors have included some firmware-based debugging features that expand beyond standard, architecturally defined capabilities of X86. For some reason though, AMD has been tightly secretive about these features; their existence was first inferred by us after considering a list of undocumented MSRs found on CBID's page (URL, cf. notes below).

Herein we uncover the outcome of our experiments, in the hope it may be useful to software developers, & possibly included in future debuggers, debugger plug-ins or other tools.

I call the new capabilities "expanded", since the term "debug extensions" is already used to refer to other features in Pentium and later processors.

Author can be contacted by email, or PM, or on the reversing forum.


_New MSRs_ :

Four undocumented machine specific registers (MSR) are involved in the expanded
debug facilities. These MSRs are "password" protected against casual access :
read/write access (RDMSR/WRMSR) to the registers is granted only if EDI holds
the correct password value, viz. EDI=9C5A203A. Otherwise, GPF exception occurs.

_Control_ @ C001_1024 , useful width: 8 bits
_Data_Match_ @ C001_1025 , width: 32
_Data_Mask_ @ C001_1026 , width: 32
_Address_Mask_ @ C001_1027 , width: 12 bits.

All four registers are zeroed upon processor reset.

Security considerations : As the features are controlled by MSRs whose access is restricted to code executed in "ring zero", their existence is generally not considered a security risk. However a malicious BIOS or OS driver could certainly make creative use of the features with some disturbing consequences against nsuspecting users.


Let's examine the _Control Register_ first :

According to the "BIOS and Kernel developer's guide" for AMD NPT Family Fh, bit 7
of this register enables an external "hardware debug tool" connected to our processor using the JTAG bus. Such (expensive, professional) tool is not considered herein.

The BIOS guide further says bits 6-0 are "reserved, should be zero".
We found that on the K7 (Athlon XP), we can put bits 1-0 to good use, as explained
below ; we have not found any effect for bits 6-2, consequently we left them aside.

We shall henceforth be discussing the use of undocumented bits 1-0 of the Control register, leaving all other bits null.


_Operational details_

The operation of breakpoint *BP0* (using DR0) is enhanced as will be described.
Breakpoints BP1 to BP3 are _not_ affected.

Breakpoint *BP0* _is_ modified, being further conditionned by the contents of the new MSRs in addition to legacy DR7. The features *cannot be switched off* : as soon as the address in DR0 is validated by setting DR7 bits 0 and/or 1, it behaves as will be explained, there is no further enabling bit.


1) The Mask MSRs :

The "Address_Mask" qualifies the Address in DR0, while "Data_Mask" qualifies the "Data_Match" MSR.

In both masks, bits which are _set_ (=1) mean "don't care", don't look at the
corresponding bit when doing compares.

A mask value of all zeroes thus is asking for exact match.
Conversely, with a data mask of all ones, comparisons will always succeed.

The Address_Mask _should_ be a string of zeroes terminated by (zero or more) ones,
in other words a power of two minus one.

Address_Mask is only twelve bits wide, hence the largest allowable address mask : 00000FFF, matches 4096 page-aligned, consecutive memory (or I/O port) addresses.


2) The Address_Mask:

It is used *unconditionally* for all three types of BP : instruction execution,
memory or IO data access.

A null mask, which is the default, in effect switches address expansion off, mimicking legacy breakpoint behavior.


3) Instruction breakpoints (DR7 type =0):

Are triggered by instruction execution at _any_ address matching DR0 under Address_Mask.

Control_ MSR has no effect (should be zero).
Data_Match and Data_Mask are not used for this type of breakpoints.


4) Memory & I/O Data breakpoint (DR7 types 1,3 and 2):

The Addres_mask is applied to DR0 address, for monitoring 1 up to 4096 consecutive bytes.

- Case: *Control = 0* (legacy), no additional check is performed. For memory access, Break occurs either on Write only, or on All_Access, selected by the legacy breakpoint "type" bits in DR7 (bite 17-16).
Data_Match and data_mask not used (should be zero).

- For the next three cases, Data compare is always done : to in effect disable it, one must use a Data_Mask of all ones (meaning : don't care).

- Case: *Control = 2* : Breaks occur on WRITE/OUT only. Even if the DR7 type is RW,
breaks never happen on Read. Traps on Data_Match.

- Case *control = 3* : same as Control = 2 , except the data condition is reversed,
i.e. Traps on Data_NON_Match.

- Case: *Control = 1* : break on Data_Match, on WRITE/OUT only, at ANY address!
Thus Address (DR0) and Address_Mask are ignored in this case (should be zero).

Reminder: I/O breakpoints require CR4 bit 3 (DE) set.


Knowledge wants to be free !
Also listed in: X86 Internals Articles
More details: Click here for more details, images, related URLs & comments for this item! (or to update its entry)

Item name: Undocumented trick : Direct access to Physical Memory on AMD K7
Rating: 0.0 (0 votes)
Author: Czernobyl aka Czerno                        
Home URL:
Last updated:
Version (if appl.):
Direct D/L link: N/A
Description: GenericIA32 Intel architecture does not provide for direct access to *physical* memory addresses in paged, protected mode. On Athlon XP and similar AMD K7 processors, however, the undocumented MSR _C0010115_ opens a read/write window into physical memory, available in all modes at CPL zero.

For more details, please see my blog (URL below).

The Forum has a discussion of whether this trick is a theoretical vulnerability.
Also listed in: X86 Internals Articles
More details: Click here for more details, images, related URLs & comments for this item! (or to update its entry)

RSS feed Feed containing all updates and additions for this category.

RSS feed Feed containing all updates and additions for this category, including sub-categories.

No items can be added directly to this category, please rather select one of its sub-categories above to submit an item!