FLEXlm latest information by CrackZ

20/11/04 - Due to a legitimate request from WISE Software (a developer of one of the programs featured in this document), the programs name and license features have been removed and replaced with *censored* to avoid any legal issues. Note that this should not detract from the fact that FLEXlm is a lousy protection.

25-01-2005 - Whats in a SIGN? (addition).


This document discusses the reversing of 3 FLEXlm protected programs. This essay is a classic example of where 1 vendors choosing not to buy into Macrovisions 'enhanced security' aided in the discovery of a very simple (and virtually generic) technique to bypass vendors that had. Throughout this essay I use * Tip * sections to help you through.

Bitplane Imaris v4.0.3 (FLEXlm v9.0.0), v3.2 could be examined for background information.
*censored* v13.0 (FLEXlm v8.2a).
SDS 2 v6.318 (FLEXlm v8.1a).

* Tip * - Use lmtools.exe, Utilities tab (sometimes shipped with your target application or inside the FLEXlm SDK to identify the version number of the FLEXlm library used). Files with FLEXlm can also be searched with something like UltraEdit for the strings 'lmgr.lib' or 'liblmgr.a'.

I started out on this project with several pieces of background information.

Bitplane Imaris - I knew the vendor keys, seeds and feature names all from v3.1/2 (which used FLEXlm v7.0e) and none of the newer FLEXlm features, I recommend downloading the files from the Bitplane WWW archive here.

*censored* v13.0 - In my possession was a patch which apparently deprotected the program, initial analysis of the changes confirmed to me that the FLEXlm routines had not been patched, instead higher level checks had been forced.

SDS 2 v6.318 - In my possession was a 'warez released' crack and a (presumed invalid) FLEXlm license. I knew from an end-user that SDS had switched to using a new style FLEXlm SIGN= license several versions before.

I started out with Bitplane Imaris, being that I knew the vendor keys, seeds and license file format, hoping this would make it easier to identify everything that was going on. I constructed a dummy license like below (based on the format I knew in the previous version, also assuming the vendor name 'bitplane' had not changed, since a daemon of the same name ships still with the program this seems like a good assumption) :

# License for Imaris Bitplane v4.0.3

FEATURE feature_name vendor_name 1.000 permanent uncounted 123456654321 \

# End license

In the past with Imaris, a fake license had always given a friendly message box complete with the feature name the program desired and the FLEXlm error code, no longer is this the case, instead we are just booted to demo mode. We need to find _lc_checkout().

Finding _lc_checkout()

Finding _lc_checkout() (when you know how) is actually very simple, however I hacked about inside my targets inside SoftICE instead of realising the one place it had to be and would be identified by IDA, lmgr.lib, there we go good developer link in our lmgr.lib and you too can tell everyone where your checkout routine is as well. IDA does a really fantastic job of decompiling lmgr.lib (be sure to select lm_ckout.obj for the object since there are more than 60 packed inside, you can examine the others later if you are so inclined). Inside you find the string reference 'lm_ckout.c', this cross references pretty much exactly (right down to the offsets of each part of the routine) with the checkout code found in all 3 of my targets. From here its a short step to identifying the real call doing the work _l_checkout().

* Tip * Note you can generate signatures from lmgr.lib using IDA's FLAIR tools, however the signatures are not without faults and often miss key functions, _l_sg() will almost certainly be missed due to its similarity to the redundant _l_svk() which GlobeTrotter leave in for obfuscation. I recommend using my 'lmgr.lib in IDA' approach.

From here we get the unsurprising version number (4.0) and the feature name ImarisBase as well as a job structure. Looking beneath _l_checkout() (inside the disassembly of lmgr.lib) we can quickly trace down an old friend, _l_sg(). Currently _l_checkout() refuses our fake license with the error -8 (the infamous LM_BADCODE). Using Hiew or any other good hex editor we do a search for F8 FF FF FF (-8 in reverse byte order) to see likely places this is set.

As it turns out we end up with 3 locations; in Imaris these are 5F7282, 5F77D7 & 5F8AE5, the first 2 are referenced inside _l_good_lic_key(), the latter inside _l_ckout_ok(), we could patch these but for now lets have a trace through _l_checkout(), the initial order of events is as follows :

_lm_start_real() - returns -8 (we'll trace below here).

So it seems _lm_start_real() is where the real action is at, note also this code snippet :

mov _lm_start$S23038, offset _lm_start_real ; lm_start = lm_start_real

FLEXlm aficionados will remember this convention from the old days of _l_sg(). Next up :

_l_valid_version() - check for valid version.
_l_clear_error() - clears the error from _l_ckout_borrow().
_l_good_lic_key() - sets the -8 error.

Once again, we get some trivial setup code, configuration verification and then _l_good_lic_key() sets our error.

_l_xor_name(bitplane) - l_xorname(job->vendor, &vc);
_l_sg() - l_sg(job, job->vendor, &vc); (did you know that l_sg means "signature vendor_key5" ;-) ).

So our old friend _l_sg() is here, in order to verify whether anything had really changed I decided to apply Nolan Blenders technique of seed recovery.

:005F6F7D lea ecx, [ebp+var_280]
:005F6F83 push ecx <-- Vendor code structure
:005F6F84 mov edx, [ebp+arg_0]
:005F6F87 add edx, 30Ch
:005F6F8D push edx <-- Vendor name
:005F6F8E mov eax, [ebp+arg_0]
:005F6F91 push eax <-- Job structure
:005F6F92 call _l_sg

Before tracing over _l_sg(), be sure to note a copy of the vendor code structure, in my case it was like so, note also that the job structure is probably not filled in :

04 00 00 00 19 59 D5 7A ED A3 2D 80 ED 11 A0 18
97 E1 4B 27 A8 21 6A E2 41 04 58 52 09 00 00 00

After tracing over _l_sg(), we observe the following changes :

Vendor code structure

04 00 00 00 52 EE F9 99 A6 14 01 63 ED 11 A0 18
97 E1 4B 27 A8 21 6A E2 41 04 58 52 09 00 00 00

Job structure

66 00 00 00 EA 00 AA 00 C6 E0 F4 21 93 65 B8 00
00 00 6F 00
00 00 00 00 00 00 00 00 02 00 00 00

Plugging these values into calcseed.exe we successfully derive 0x0A192837 (seed 1) & 0xF0E1D2C3 (seed 2), these match the seeds from the previous version. Generating vendor keys using an updated version of lmkg.exe (courtesy of good friend Nolan Blender or using lmv8gen.exe (found somewhere on my FLEXlm page)) we can generate licenses (note that you'll need to insert defines for ENCRYPTION_SEED1 & ENCRYPTION_SEED2 or copy them from an older lm_code.h), I also found in later SDK's LM_SEED's must also be defined, although they won't be used if you set LM_STRENGTH appropriately.

Bitplane will now check this license out successfully, this indicates that it does not use any of the new enhanced FLEXlm features even though all of the routines are not compiled out, you can now freely generate licenses for the remainder of the features, accessible from the very convenient license option.

A Snag

FLEXlm couldn't be this easy?, surely?, lets try the technique above with *censored*.....As it turns out *censored* tries to check out 7 features :

*censored* / *censored* / *censored* / *censored* / *censored* / *censored* / *censored* (*censored* & *censored* are checked out by ODB2GT.exe & PADS2GT.exe respectively)

I was informed each of these corresponds to an edition type of *censored*, so only 1 feature should actually be checked out successfully. I'd generated my license for *censored* and _l_checkout() rejected it with error -8 (LM_BADCODE), the other features failed with -5 (LM_NOFEATURE), no real surprise there, crucially we now know there must therefore be something else beneath _l_checkout() that determines that the license is invalid.

I was pretty confident that the reason the license was rejected was due to the new ECC security, since this has been well implemented (Certicom did their job correctly), there is no hope of finding the real LM_SEED's. A brute force attack would be 2^96, inpracticle to say the least (and rendered further so by the speed of the authentication code). There are now 2 things to do, first we can probably tease from the FLEXlm library the -8 error (0xFFFFFFF8) and find where its set (like Imaris, a hex editor will do for finding those places in *censored*), as it turns out there are 6 address candidates, simple testing would quickly find the correct one. Or, secondly, we can just trace on from _l_sg() with *censored*.

Tracing onwards I simply compared the code flow from Imaris with that of *censored*, both targets next call _l_ckout_crypt() and code flow differs soon after a call to _real_crypt(), modifying execution at this level produces a null pointer error. With Imaris code flow is directed towards a familiar side-by-side comparison of the real license with the one obtained from the license file, this implies that _real_crypt() and routines below must be deciding whether to perform the old style checkout or the new style authentication. Lets look at the piece of code that controls this (from Imaris) :


:005F9AF0 push ebp
:005F9AF1 mov ebp, esp
:005F9AF3 sub esp, 9ACh
:005F9AF9 lea eax, [ebp+var_93C]
:005F9AFF mov [ebp+var_4], eax
:005F9B02 mov [ebp+var_948], 0
:005F9B0C mov [ebp+var_960], 0
:005F9B16 mov [ebp+var_950], 0
:005F9B20 mov [ebp+var_968], 0
:005F9B2A mov ecx, [ebp+arg_4]
:005F9B2D mov edx, [ecx+50h]
:005F9B30 mov [ebp+var_964], edx
:005F9B36 mov eax, [ebp+arg_8]
:005F9B39 mov [ebp+var_944], eax
:005F9B3F mov [ebp+var_96C], 0
:005F9B49 mov [ebp+var_940], 0
:005F9B53 mov ecx, [ebp+arg_0]
:005F9B56 mov edx, [ecx+3FCh] ; Type of license to checkout
:005F9B5C and edx, 100000h
:005F9B62 test edx, edx
:005F9B64 jnz short loc_5F9B72

The value of EDX at 5F9B56 controls which type of license 'check out' will be performed, with Imaris the value was 0x1048C0 and with *censored* 0x48C0, if we now modify this value in a live debugging session, *censored* will check out using the old style FLEXlm encryption and we'll be licensed successfully. The next step is to see where the FLEXlm routines set this flag. With a simple bpm we can backtrace our value being set using an 'or ecx, 100000h' instruction inside _l_good_lic_key(), this is referenced back from another flag checked just after _l_sg().

:005F6F92 call _l_sg
:005F6F97 add esp, 0Ch
:005F6F9A mov ecx, [ebp+arg_0]
:005F6F9D cmp dword ptr [ecx+51Ch], 0
:005F6FA4 jz loc_5F704A <-- if ((job->L_SIGN_LEVEL) && !conf->lc_keylist).

So if we've got a 0 here we perform the old checkout, we could now just settle for patching here but lets backtrace yet again. This results in 2 locations that write 1 to our [ecx+51Ch] flag, inside *censored* these are 6AF454 (at the end of _l_init()) and 6B219C (at the end of _lc_new_job()).

Flag 1

6AF454 - Set to 1 using a static ADD ECX, 1 instruction at 6AF426, this could be patched to ADD ECX, 0 for a 1 byte change however Imaris has the same reference so there must be a variation in code path. By comparing the flow between Imaris and *censored*, we discover the following 'switching' code :

:006AF354 mov edx, [ebp+var_5DC]
:006AF35A cmp dword ptr [edx], 0
:006AF35D jz loc_6AF434 ; 0x0 Imaris, 0x10 *censored*

This is the real switch we have to backtrace and its set deep inside the _l_buf_36() (routine described below), the only way I reliably found to locate where the static value is stored is as follows.

i). Breakpoint the _l_buf_36() routine, do d *(esp+8) to display in the data window the pointer to the vendor code structure, in some instances you may need to pagein this address via SoftICE.

ii). Set a bpm w on [vendor code structure + 3Ch] and monitor writes, anything other than zero should yield the static location of the data being written there (note that you'll probably get 3 or so breaks on access before finding the right one), once we've located the correct place we can make a small patch of the static data.

:00405FB6 mov eax, [edx+3Ch]
:00405FB9 add eax, dword_81C7F8 ; 0x10 static data
:00405FBF mov ecx, [ebp+0Ch]
:00405FC2 mov [ecx+3Ch], eax ; Write flag

Flag 2

6B219C - Set via function call at the start of _lc_new_job(). See following code :

_lc_new_job proc near
:006B20B0 push ebp
:006B20B1 mov ebp, esp
:006B20B3 sub esp, 14h
:006B20B6 mov eax, _l_n36_buf
:006B20BB mov [ebp+arg_4], eax
:006B20BE lea ecx, [ebp+var_10]
:006B20C1 push ecx
:006B20C2 push 0
:006B20C4 push 0
:006B20C6 push 0
:006B20C8 mov edx, [ebp+arg_8]
:006B20CB push edx
:006B20CC lea eax, [ebp+var_C]
:006B20CF push eax
:006B20D0 call [ebp+arg_4] ; _l_n36_buf()

_l_n36_buf dd 401000h

:0040100F mov eax, [ebp+1Ch]
:00401012 mov dword ptr [eax], 1 ; Set flag (we want 0)

This requires a patch, in Imaris this flag is AND'd with 0, *censored* sets it to 1 & SDS to 2. If we make the necessary changes *censored* will launch and license successfully. I've developed a 'muster' which you might care to print and apply to your next FLEXlm target. Lets apply my technique to SDS 2 v6.318. (Please note that this method should not be totally relied upon as a quick substitute to commenting your IDA database using lmgr.lib), the screenshots below are taken from a fully commented .idb.

* Tip * It can be worthwhile loading lm_crstr.obj in IDA to aid commenting of your target idb.

1. Locate _l_sg().

This seems to be trivial by searching for the hex constant 6F7330B8h (unsigned long x = 0x6f7330b8; /*- v8.x */) used at the start of the routine, (note that there are 2 references to this constant and it is the first one you find which is _l_sg(), the other is inside _l_svk() purely for obfuscation). If you feel so inclined you can label dword_161DEC4 as _l_n36_buff.

_l_sg = sub_DBFD0A

2. Locate _l_init().

_l_sg() should have 6 cross references. By quickly examing the code immediately after each call to _l_sg() looking for the check against the default seeds 0x12345678 & 0x87654321 _l_init() can be identified.

:00DCD13E call _l_sg
:00DCD143 add esp, 0Ch
:00DCD146 cmp dword ptr [ebp-27Ch], 87654321h
:00DCD150 jz short loc_DCD15E
:00DCD152 cmp dword ptr [ebp-278h], 12345678h
:00DCD15C jnz short loc_DCD1BB
:00DCD15E loc_DCD15E: ; CODE XREF: sub_DCC520+C30j

_l_init() = sub_DCC520

3. Locate _l_good_lic_key().

By examining the other references to _l_sg(), _l_good_lic_key() will be the only reference that calls _l_sg() twice, in my fully commented .idb this is clear to see.

_l_good_lic_key = sub_DBE15D

4. _l_good_lic_key() has 9 xrefs of the form 3 xrefs inside 3 functions, the top most of these in the library is _lm_start_real(), again in a fully commented .idb, this is clear to see.

_lm_start_real() = sub_00DBD112

The only xref to _lm_start_real() should be inside _l_checkout().

_l_checkout() = sub_DBCA0E

From _l_checkout() we can easily deduce _lc_checkout() by finding the reference with lm_ckout.c at the start of the function (the other reference to _l_checkout() is inside _l_reconnect()). _lc_checkout() = sub_DBC940

5. _lc_checkout() has 2 cross references, the 2nd of which is of the form mov dword ptr [reg32+30h], offset _lc_checkout() (this is actually part of _InitLmInterface()), by scrolling up here we can label the routine at [reg32+0Ch] (or the first entry) as _lc_init(), this yields _lc_new_job() which call's _lc_init() as sub_DC8610 and _l_n36_buf as dd 0E07100h.

*Tip* You can label all of the functions from _InitLmInterface() by loading lmgr.lib into IDA and selecting l_lock_load.obj. In later FLEXlm versions lc_auth_data() which might well be used by a target can be found in l_check.obj.

6. In a live debugging session we set breakpoints on :

_l_n36_buf = E07100
_l_sg = DBFD3D
_lc_checkout = DBC99D

We construct a dummy license file, like so (note that you probably ought to be able to ascertain the name of the vendor daemon) :

FEATURE somefeature dsndata 1.0 permanent uncounted 0 \

From the break inside _l_n36_buf() we discover that our 2 bad flags are set at address E07112 & E0D4BD. From the break inside _lc_checkout(), we get the feature name sds2 and the version number 6.318, lets amend the license accordingly.

FEATURE sds2 dsndata 6.318 permanent uncounted 0 \

* Tip * Most FLEXlm targets vendor daemons don't use the _l_n36_buf() configuration flags, this can make them an easy way to recover seeds using lmgrd -z daemon_name -c license file, especially if you don't get a break on _l_sg() in your regular target or _l_checkout() returns EAX=-15.

You should now get a break on _l_sg() and the call to the _l_n36_buff() decoding routine, this should enable you to recover the seeds (0x987AC78F & 0xC8D6382B). Using lmv8gen you can generate vendor keys and now compile lmcrypt.exe. This will enable you to generate a valid license.

* Tip * By leaving your breakpoint on _lc_checkout() active you should be able to recover any other feature names checked out by your target application and thus generate licenses for them.

With SDS 2 the approach I describe above lets _lc_checkout() return 0, however the program crashes unexpectedly afterwards; this was puzzling since the warez released version simply patches sub_DBC940 (what we know to be _lc_checkout() to XOR EAX, EAX / RETN), exactly the same effect as our license. Also the license with the release was evidently generated using the same key information I recovered above (as lmcrypt didn't change it). SDS 2 appears to be an anomalous case amongst FLEXlm targets, I tried 3 others and couldn't replicate this behaviour, however its useful to know that we can also use this somehow more 'brutal technique' of patching _lc_checkout(). Since there is also integrity checking of the file, patches were also made to static checksum data, this was implemented separately by the developers obviously to prevent file tampering.


I hadn't really looked at FLEXlm for quite some time and Macrovision are to be commended for finally making a protection which is safe from key generators, however, whilst they continue to support their legacy mistakes, old style FLEXlm licenses can still be generated and the application trivially patched to make the licensing layer accept them, even if my technique was to be broken, it is still far too easy to patch the _lc_checkout() API directly.

Whats in a SIGN?

The tutorial above describes a mechanism by which we can patch the FLEXlm licensing layer to use the license key checkout (this is the oldest and simplest of the FLEXlm validation methods and was chosen purely for simplicity). The basic SIGN attribute was added by Globetrotter at around v7.x (it has only 12 chars) and offers merely an improved algorithm (perhaps more resistance to brute forcing) and better seed hiding, these are about the only 'enhancements' if one wishes to call them that.

These days a lot of customers have switched to using the CRO or TRO (counterfeit/tamper resistant options), really the same thing under a new name. This offers customers the ability to generate the newest style ECC SIGN licenses with strings starting at 58 chars, as far as I know and can verify there have been no successful attacks against ECC FLEXlm which enables either complete recovery of the private key or the LM_SEED's (I do not rule out however that there is enough processing power somewhere to recover them).

The new SIGN length has resulted in most crackers choosing to patch _lm_pubkey_verify() and generating a SIGN= license using their own LM_SEED's. There is however an alternative approach which involves forcing the licensing layer to do the old style SIGN=12 chars checkout and it simply involves patching only the 2nd bad flag we found inside _l_n36_buf() (see above). We can then recover the encryption seeds as before and generate a license using either the SDK or Lmcryptgui available on this site. The patch works by simply telling the licensing layer not to get the address of _lm_pubkey_verify() which is checked shortly after _l_sg().

Another important thing to note, its easy to verify if your target will allow the old style SIGN= checkout, after _l_sg() set a breakpoint on the mangled seeds in the vendor code structure, if it hits the seeds are being recovered and you can generate the old style standard SIGN, if not, you'll need a patch; after a patch a breakpoint on the mangled seeds should hit, just prior to them being recovered.

Return to FLEXlm Return to Main Index

© 1998-2005 CrackZ. January 21st 2005.