WRKRazzle

Any off topic discussions should go in this forum. Post count is not increased by posting here.
FTP Access status is required to post in this forum. Find out how to get it
Forum rules
Any off topic discussions should go in this forum. Post count is not increased by posting here.
FTP Access status is required to post in this forum. Find out how to get it
Post Reply
Stephanos
Posts: 297
Joined: Fri Dec 26, 2014 5:38 am
Contact:

WRKRazzle

Post by Stephanos »

WRKRazzle

Project Website: http://opennt.net/projects/wrkrazzle/wiki
SVN Repository: svn://svn.opennt.net/WRKRazzle/trunk
Zipped Repository: http://opennt.net/attachments/download/ ... ort.r29.7z

Summary

The WRKRazzle is a repackaged version of the Windows Research Kernel v1.2 that is compatible with the NTOSBE Razzle environment.

As compared to the standard Windows Research Kernel distribution, WRKRazzle offers the following advantages:
  • Multiprocessor build is supported through the magic of the Razzle Build Utility (build.exe).
  • Kernel (base\ntos\ke) service table is automatically generated from service.tab.
  • Runtime (base\ntos\rtl) error.h is automatically generated from generr.c.
  • The build infrastructure is much cleaner than the hack-job makefile build environment provided with the WRK.
  • Razzle is what Microsoft internally uses.
Setting up the Build Environment

WRKRazzle is intended to be built with the NTOSBE. NTOSBE is the Razzle build environment developed and used by the OpenNT Project. Its main use is for building the NT 4.5, but it may also be used for building the WRKRazzle.

1. Check out the NTOSBE repository. The local NTOSBE directory may be located anywhere, but it is recommended that you place this directory under the drive root (e.g. W:\NTOSBE) for the sake of simplicity.

Code: Select all

svn co svn://svn.opennt.net/NTOSBE/trunk W:\NTOSBE
2. Check out the WRKRazzle repository. This repository may also be located anywhere on your local drive, but it is recommended that you place it under the drive root.

Code: Select all

svn co svn://svn.opennt.net/WRKRazzle/trunk W:\WRKRazzle
3. While the WRKRazzle repository is being checked out, go to the NTOSBE directory and open siz_templ.cmd in your favourite text editor. You will see something like the following:

Code: Select all

@echo off

REM //
REM // SIZ_NTROOT: Path to the NT 4.5 source root (e.g. W:\trunk.x86chk)
REM //

set SIZ_NTROOT=W:\trunk.x86chk

REM //
REM // SIZ_NTTREE: Path to the NT 4.5 binary root (e.g. W:\bin)
REM //

set SIZ_NTTREE=W:\bin

REM //
REM // SIZ_NTARCH: Target build architecture [x86, amd64, arm, mips, alpha, ppc]
REM //

set SIZ_NTARCH=x86

REM //
REM // SIZ_NTBLD: Target build type [fre, chk]
REM //

set SIZ_NTBLD=chk

REM //
REM // Repository-specific Parameters
REM //

REM //
REM // _NT_TARGET_VERSION: If the target repository version is not 4.5, set the following value.
REM //                     For building WRKRazzle, this value must be set to 0x0502 (5.2).
REM //

REM set _NT_TARGET_VERSION=0x0502

REM //
REM // Call sizzle
REM //

call sizzle.cmd
4. Set SIZ_NTROOT to the WRKRazzle directory (in our demo, W:\WRKRazzle) and SIZ_NTTREE to the binary output path you want (e.g. W:\WRKRazzle.bin). When successfully built, the ntkrnlmp.exe will be 'binplace'd under SIZ_NTTREE\(TARGETNAME), where (TARGETNAME) would be something like x86fre or amd64fre.

5. Set SIZ_NTARCH to your desired target architecture. For WRK, x86 and amd64 target architectures are supported.

6. Set SIZ_NTBLD to fre. For WRK, only Free (fre) build is supported because the included prebuilt libraries are 'Free'.

7. Uncomment _NT_TARGET_VERSION. Make sure _NT_TARGET_VERSION is set to 0x0502.

8. Save as siz_WRKRazzle.%SIZ_NTARCH%%SIZ_NTBLD%.cmd (e.g. siz_WRKRazzle.x86fre.cmd).

NOTE: You may create as many siz_*.cmd script files as you need. For example, you may have both siz_WRKRazzle.x86fre.cmd and siz_WRKRazzle.amd64fre.cmd for building x86fre and amd64fre, respectively.

The following is an example siz_WRKRazzle.x86fre.cmd script file:

Code: Select all

@echo off

REM //
REM // SIZ_NTROOT: Path to the NT 4.5 source root (e.g. W:\trunk.x86chk)
REM //

set SIZ_NTROOT=W:\WRKRazzle

REM //
REM // SIZ_NTTREE: Path to the NT 4.5 binary root (e.g. W:\bin)
REM //

set SIZ_NTTREE=W:\WRKRazzle.bin

REM //
REM // SIZ_NTARCH: Target build architecture [x86, amd64, arm, mips, alpha, ppc]
REM //

set SIZ_NTARCH=x86

REM //
REM // SIZ_NTBLD: Target build type [fre, chk]
REM //

set SIZ_NTBLD=fre

REM //
REM // Repository-specific Parameters
REM //

set _NT_TARGET_VERSION=0x0502

REM //
REM // Call sizzle
REM //

call sizzle.cmd
p.s. Why sizzle.cmd (when it should supposedly be called razzle.cmd)? To be honest, I don't know. I just decided to name it that. After all, it will make your processor sizzle once you start a full build, so the nomenclature isn't totally incorrect.

Building the WRKRazzle

Before attempting to build the WRKRazzle, you must have carefully followed the set up procedure described in the "Setting up the Build Environment" section. If not, please go over them and make sure that you have correctly set up the build environment.

1. Open the NTOSBE by clicking the siz_*.cmd script (e.g. siz_WRKRazzle.x86fre.cmd).
2. Type 'build' to perform a normal build, 'build -c' to perform a clean build.

That's it.

Depending on your processor performance, you should have the kernel executable in the binary path you specified in your siz_*.cmd script (SIZ_NTTREE) in tens of seconds.

Limitations

These limitations are inherent to the Windows Research Kernel, not the WRKRazzle.
  • Checked build is not supported.
  • Only x86 and amd64 processor architectures are supported.
  • Uniprocessor build (up) is not supported.
  • PAE build is not supported (mipae.h is missing).
  • genxx is not performed (genxx input files are missing; though, this should be relatively easy to reconstruct). NOTE: genxx generates ks* and hal* asm includes that list kernel struct offsets for use in asm code.
Future Plans

There is no plan for new feature implementations in this project. WRKRazzle will stay as the pure Windows Research Kernel with a few minor modifications to support building under the Razzle environment.

This project may be forked for other OpenNT research projects that may involve source code reconstruction and feature implementations tasks.

Image

Image

Image
Last edited by Stephanos on Thu Jun 11, 2015 4:07 am, edited 2 times in total.

Samuka
FTP Access
Posts: 535
Joined: Tue Nov 10, 2009 4:31 am
Location: Brazil
Contact:

Re: WRKRazzle

Post by Samuka »

Windows 2000 source code has a mipae.h. Only minor things are missing, you can start by it

wasabilee
Permanently Banned
Posts: 491
Joined: Tue Jun 10, 2014 5:09 pm

Re: WRKRazzle

Post by wasabilee »

so this is basically recompiled NT 4

Stephanos
Posts: 297
Joined: Fri Dec 26, 2014 5:38 am
Contact:

Re: WRKRazzle

Post by Stephanos »

Samuka wrote:Windows 2000 source code has a mipae.h. Only minor things are missing, you can start by it
Yes, I am aware of it. There has been quite a bit of changes in mm from NT 5 to NT 5.2, so NT 5 mipae.h cannot be directly used. It will take a significant amount of reverse engineering to fully reconstruct NT 5.2 mipae.h. Although I have no plans for doing that for this project as I mentioned (I will keep this as pure WRK), I might do it later in forks of this project.

p.s. I created this project because I need a known working test platform for the kernel power manager I'm implementing in NT 4.5 trunk.
Last edited by Stephanos on Thu Jun 11, 2015 2:59 am, edited 1 time in total.

Stephanos
Posts: 297
Joined: Fri Dec 26, 2014 5:38 am
Contact:

Re: WRKRazzle

Post by Stephanos »

wasabilee wrote:so this is basically recompiled NT 4
This is the Windows Research Kernel (NT 5.2) with makefile build environment removed and (reconstructed) Razzle environment added back (it would originally have been Razzle, but Microsoft replaced it with a hacky/dumbed down version).

yourepicfailure
User avatar
Donator
Posts: 1234
Joined: Mon Jul 23, 2012 9:40 pm
Location: Flying high in a DC10

Re: WRKRazzle

Post by yourepicfailure »

Stephanos wrote:
wasabilee wrote:so this is basically recompiled NT 4
This is the Windows Research Kernel (NT 5.2) with makefile build environment removed and (reconstructed) Razzle environment added back (it would originally have been Razzle, but Microsoft replaced it with a hacky/dumbed down version).
Ms probably dumbed it down so people didn't do "funky" things with the environment.
What those "funky" things may be, well it's up to anyone. Maybe usage in the NT4 and win2k code?
Considering 10% of the WRK code is precompiled by MS before distribution.

Now if someone had the IE build environment...
"C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off"
Image
You will never tear me from the grasp of the Pentium M!

Stephanos
Posts: 297
Joined: Fri Dec 26, 2014 5:38 am
Contact:

Re: WRKRazzle

Post by Stephanos »

yourepicfailure wrote:
Stephanos wrote:
wasabilee wrote:so this is basically recompiled NT 4
This is the Windows Research Kernel (NT 5.2) with makefile build environment removed and (reconstructed) Razzle environment added back (it would originally have been Razzle, but Microsoft replaced it with a hacky/dumbed down version).
Ms probably dumbed it down so people didn't do "funky" things with the environment.
What those "funky" things may be, well it's up to anyone. Maybe usage in the NT4 and win2k code?
Considering 10% of the WRK code is precompiled by MS before distribution.

Now if someone had the IE build environment...
I suppose that could have been one of their intentions.

There is no such thing as "IE build environment" nor should there be a need for one. IE, along with pretty much everything else that doesn't use MSBuild, is built using Razzle.

yourepicfailure
User avatar
Donator
Posts: 1234
Joined: Mon Jul 23, 2012 9:40 pm
Location: Flying high in a DC10

Re: WRKRazzle

Post by yourepicfailure »

Stephanos wrote:There is no such thing as "IE build environment" nor should there be a need for one.
Take a look at the Win2k code. It's required to compile Trident.
However, it's not related to this topic. Perhaps I should start one exclusively for discussion of such topic?
"C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off"
Image
You will never tear me from the grasp of the Pentium M!

Stephanos
Posts: 297
Joined: Fri Dec 26, 2014 5:38 am
Contact:

Re: WRKRazzle

Post by Stephanos »

yourepicfailure wrote:
Stephanos wrote:There is no such thing as "IE build environment" nor should there be a need for one.
Take a look at the Win2k code. It's required to compile Trident.
However, it's not related to this topic. Perhaps I should start one exclusively for discussion of such?
No need for further discussion. mshtml build also uses Razzle. It just requires a few extra environment variables set.

yourepicfailure
User avatar
Donator
Posts: 1234
Joined: Mon Jul 23, 2012 9:40 pm
Location: Flying high in a DC10

Re: WRKRazzle

Post by yourepicfailure »

Stephanos wrote:No need for further discussion. mshtml build also uses Razzle. It just requires a few extra environment variables set.
And a few file dependencies resolved.
"C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off"
Image
You will never tear me from the grasp of the Pentium M!

Stephanos
Posts: 297
Joined: Fri Dec 26, 2014 5:38 am
Contact:

Re: WRKRazzle

Post by Stephanos »

I had to generate the following list for something I am working on, and decided to post it in case anyone needs it.

Full list of kernel source files that are NOT included in Windows Research Kernel distribution (only available as binary form in prebuilt library):

Code: Select all

base\ntos\arb\arbiter.c
base\ntos\arb\debug.c
base\ntos\cache\prefboot.c
base\ntos\cache\prefetch.c
base\ntos\cache\prefparm.c
base\ntos\config\amd64\geninst.c
base\ntos\config\amd64\initamd64.c
base\ntos\config\amd64\initdat.c
base\ntos\config\amd64\parseini.c
base\ntos\config\amd64\rules.c
base\ntos\config\cmapi3.c
base\ntos\config\cmdat3.c
base\ntos\config\cmsysini2.c
base\ntos\config\hwprofil.c
base\ntos\config\i386\geninst.c
base\ntos\config\i386\init386.c
base\ntos\config\i386\initdat.c
base\ntos\config\i386\parseini.c
base\ntos\config\i386\rules.c
base\ntos\ex\dbgctrlex.c
base\ntos\ex\exhotp.c
base\ntos\ex\exinfo.c
base\ntos\ex\exscan.c
base\ntos\ex\hdlsterm.c
base\ntos\ex\sysenv.c
base\ntos\ex\systime.c
base\ntos\fstub\ex.c
base\ntos\fstub\halfnc.c
base\ntos\fstub\translate.c
base\ntos\init\anim.c
base\ntos\init\bootvid.c
base\ntos\init\init.c
base\ntos\init\port.c
base\ntos\io\iomgr\arcsec.c
base\ntos\io\iomgr\dumpctl.c
base\ntos\io\iomgr\ioperf.c
base\ntos\io\iomgr\ioverifier.c
base\ntos\io\iomgr\query.c
base\ntos\io\iomgr\triage.c
base\ntos\io\iovutil.c
base\ntos\io\netboot.c
base\ntos\io\pnpmgr\assign.c
base\ntos\io\pnpmgr\devices.c
base\ntos\io\pnpmgr\devintrf.c
base\ntos\io\pnpmgr\devnode.c
base\ntos\io\pnpmgr\dockhwp.c
base\ntos\io\pnpmgr\i386\pbiosa.asm
base\ntos\io\pnpmgr\i386\pbiosc.c
base\ntos\io\pnpmgr\iofileutil.c
base\ntos\io\pnpmgr\mapper.c
base\ntos\io\pnpmgr\pnpbusno.c
base\ntos\io\pnpmgr\pnpcvrt.c
base\ntos\io\pnpmgr\pnpdata.c
base\ntos\io\pnpmgr\pnpdd.c
base\ntos\io\pnpmgr\pnpdel.c
base\ntos\io\pnpmgr\pnpdma.c
base\ntos\io\pnpmgr\pnpeisa.c
base\ntos\io\pnpmgr\pnpenum.c
base\ntos\io\pnpmgr\pnpevent.c
base\ntos\io\pnpmgr\pnpinit.c
base\ntos\io\pnpmgr\pnpioapi.c
base\ntos\io\pnpmgr\pnpirp.c
base\ntos\io\pnpmgr\pnpirq.c
base\ntos\io\pnpmgr\pnpmap.c
base\ntos\io\pnpmgr\pnpmemio.c
base\ntos\io\pnpmgr\pnppower.c
base\ntos\io\pnpmgr\pnpres.c
base\ntos\io\pnpmgr\pnprlist.c
base\ntos\io\pnpmgr\pnpstart.c
base\ntos\io\pnpmgr\pnpsubs.c
base\ntos\io\pnpmgr\ppcddb.c
base\ntos\io\pnpmgr\ppcontrol.c
base\ntos\io\pnpmgr\ppdrvdb.c
base\ntos\io\pnpmgr\pphandle.c
base\ntos\io\pnpmgr\pphotswap.c
base\ntos\io\pnpmgr\pplastgood.c
base\ntos\io\pnpmgr\pppagepath.c
base\ntos\io\pnpmgr\ppprofile.c
base\ntos\io\pnpmgr\ppvutil.c
base\ntos\io\pnpmgr\remlock.c
base\ntos\io\pnpmgr\report.c
base\ntos\io\sessnirp.c
base\ntos\io\trackirp.c
base\ntos\kd64\amd64\kdcmsup.c
base\ntos\kd64\amd64\kdcpuapi.c
base\ntos\kd64\dbgctrl.c
base\ntos\kd64\file.c
base\ntos\kd64\i386\kdcmsup.c
base\ntos\kd64\i386\kdcpuapi.c
base\ntos\kd64\i386\kdtrap.c
base\ntos\kd64\kdapi.c
base\ntos\kd64\kdbreak.c
base\ntos\kd64\kddata.c
base\ntos\kd64\kddbgio.c
base\ntos\kd64\kdinit.c
base\ntos\kd64\kdlock.c
base\ntos\kd64\kdmove.c
base\ntos\kd64\kdquery.c
base\ntos\kd64\print.c
base\ntos\ke\amd64\foltz.asm
base\ntos\ke\bugchcki.c
base\ntos\ke\i386\allprochal.c
base\ntos\ke\i386\biosa.asm
base\ntos\ke\i386\cyrix.c
base\ntos\ke\i386\vdmke.c
base\ntos\mm\crashdmp.c
base\ntos\mm\debugsup.c
base\ntos\mm\dynmem.c
base\ntos\mm\mirror.c
base\ntos\mm\mmcompress.c
base\ntos\mm\mmpatch.c
base\ntos\mm\mmsan.c
base\ntos\mm\mmtriage.c
base\ntos\mm\nolowmem.c
base\ntos\mm\pfsup.c
base\ntos\mm\specpool.c
base\ntos\mm\verifier.c
base\ntos\ob\obperf.c
base\ntos\ob\obvutil.c
base\ntos\perf\amd64\setlog.c
base\ntos\perf\i386\setlog.c
base\ntos\po\amd64\xsum.asm
base\ntos\po\attrib.c
base\ntos\po\hiber.c
base\ntos\po\i386\xsum.asm
base\ntos\po\idle.c
base\ntos\po\misc.c
base\ntos\po\notify.c
base\ntos\po\ntapi.c
base\ntos\po\paction.c
base\ntos\po\pbatt.c
base\ntos\po\pidle.c
base\ntos\po\pinfo.c
base\ntos\po\pocall.c
base\ntos\po\podata.c
base\ntos\po\poinit.c
base\ntos\po\ponotify.c
base\ntos\po\poshtdwn.c
base\ntos\po\postate.c
base\ntos\po\pwork.c
base\ntos\po\sidle.c
base\ntos\po\suspend.c
base\ntos\po\switch.c
base\ntos\po\sys.c
base\ntos\po\sysdev.c
base\ntos\po\thermal.c
base\ntos\po\throttle.c
base\ntos\po\volume.c
base\ntos\rtl\amd64\lznt1amd.asm
base\ntos\rtl\backoff.c
base\ntos\rtl\bootstatus.c
base\ntos\rtl\checksum.c
base\ntos\rtl\compress.c
base\ntos\rtl\gen8dot3.c
base\ntos\rtl\heap.c
base\ntos\rtl\i386\lzntx86.asm
base\ntos\rtl\lznt1.c
base\ntos\rtl\prodtype.c
base\ntos\rtl\random.c
base\ntos\rtl\range.c
base\ntos\rtl\rtlpatch.c
base\ntos\rtl\rxact.c
base\ntos\rtl\version.c
base\ntos\rtl\xencode.c
base\ntos\vdm\i386\rdwr.c
base\ntos\vdm\i386\strtexec.c
base\ntos\vdm\i386\vdmentry.c
base\ntos\vdm\i386\vdmfault.c
base\ntos\vdm\i386\vdminit.c
base\ntos\vdm\i386\vdmint21.c
base\ntos\vdm\i386\vdmints.c
base\ntos\vdm\i386\vdmnpx.c
base\ntos\vdm\i386\vdmop0f.asm
base\ntos\vdm\i386\vdmoprnd.asm
base\ntos\vdm\i386\vdmprint.c
base\ntos\vdm\i386\vdmtib.c
base\ntos\vdm\i386\vdmtrace.c
base\ntos\vdm\i386\vdmuserrefs.c
base\ntos\vdm\vdm.c
base\ntos\verifier\vfbugcheck.c
base\ntos\verifier\vfddi.c
base\ntos\verifier\vfdeadlock.c
base\ntos\verifier\vfdebug.c
base\ntos\verifier\vfdevobj.c
base\ntos\verifier\vffileio.c
base\ntos\verifier\vffilter.c
base\ntos\verifier\vfgeneric.c
base\ntos\verifier\vfhal.c
base\ntos\verifier\vfinit.c
base\ntos\verifier\vfirp.c
base\ntos\verifier\vfirpdb.c
base\ntos\verifier\vfirplog.c
base\ntos\verifier\vfmajor.c
base\ntos\verifier\vfmessage.c
base\ntos\verifier\vfpacket.c
base\ntos\verifier\vfpnp.c
base\ntos\verifier\vfpower.c
base\ntos\verifier\vfprint.c
base\ntos\verifier\vfrandom.c
base\ntos\verifier\vfsettings.c
base\ntos\verifier\vfstack.c
base\ntos\verifier\vftriage.c
base\ntos\verifier\vfutil.c
base\ntos\verifier\vfwmi.c
base\ntos\verifier\vfzwapi.c

yourepicfailure
User avatar
Donator
Posts: 1234
Joined: Mon Jul 23, 2012 9:40 pm
Location: Flying high in a DC10

Re: WRKRazzle

Post by yourepicfailure »

yourepicfailure wrote:Considering 10% of the WRK code is precompiled by MS before distribution.
Would've been great to have had a little more HAL code, other than some debugging files and libraries to have to decompile.
"C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off"
Image
You will never tear me from the grasp of the Pentium M!

Stephanos
Posts: 297
Joined: Fri Dec 26, 2014 5:38 am
Contact:

Re: WRKRazzle

Post by Stephanos »

BUGFIX 2015-08-09 (WRKRazzle Project Bug #229)

Symptom: Modifying "Write debugging information" properties in System Properties-Startup and Recovery-Settings will result in a BSOD.

Analysis: When the system crash dump settings are modified, the following system calls are issued:

Code: Select all

NtSetSystemInformation(SystemCrashDumpStateInformation) -> IoConfigureCrashDump -> IoInitializeCrashDump -> IoGetDumpStack -> IopGetDumpStack -> KeGetBugMessageText
KeGetBugMessageText function references the global pointer variable KiBugCodeMessages (declared in kernldat.c). This variable is initialised by the ExpInitializeExecutive function as follows:

Code: Select all

        Status = LdrFindResource_U (DataTableEntry->DllBase,
                                    ResourceIdPath,
                                    3,
                                    (VOID *) &ResourceDataEntry);
 
        if (NT_SUCCESS(Status)) {
 
            Status = LdrAccessResource (DataTableEntry->DllBase,
                                        ResourceDataEntry,
                                        &MessageData,
                                        NULL);
 
            if (NT_SUCCESS(Status)) {
                KiBugCodeMessages = MessageData;
            }
        }
Note that the KiBugCodeMessage pointer obtained using LdrAccessResource function will reside in the resource (.rsrc) section.

By default, the resource section is marked as discardable and this was causing the KiBugCodeMessage address to be invalid after kernel initialisation, causing a PAGE_FAULT_IN_NONPAGED_AREA bugcheck when calling KeGetBugMessageText.

Resolution: The following linker flag has been added to the linker options when linking the kernel executable:

Code: Select all

-section:.rsrc,!d
The bug fix repository revision is r31.

Post Reply