ciforth - The LAB library mechanism

ciforth (common Intel Forth), gemaakt door Albert van der Horst, is een Forth voor Intel processoren en voldoet aan de ISO Forth standaard. Het is beschikbaar voor Linux (ook 64 bit), MS-Windows and OSX. De kleine, klassieke, indirect bedraded kernel bevat het hoogstnoodzakelijke, de CORE woorden set, string manipulaties, het benaderen van files en exceptieafhandeling. De kracht wordt vermenigvuldigd door een uitgebreide source bibliotheek, die automatisch benaderd kan worden, en onder meer een decompilator en een geïntegreerde Intel 386 assembler.

ciforth is eenvoudig, staat volledig op zichzelf en kan gemakkelijk aangepast worden; dit is dé kans om een compiler volledig te doorgronden.

Configuratiegegevens zijn strings binnen de Forth zelf. Hierdoor en door de bibliotheek zijn ciforth-applikaties overdraagbaar tussen Linux, Windows en MS-DOS.

ciforth info (author's website)

Inleiding ciforth

Verdwijnende files

Microsoft treedt soms betuttelend op en verwijdert files die U bewust neer gezet hebt. Waneer er files verdwijnen onder MS-Windows 7 of 10, volg dan het advies gegeven op

https://www.easeus.com/file-recovery/stop-windows-10-from-deleting-files-without-asking.html

Source en copyright

Dit systeem wordt beschikbaar gesteld onder het zg GPL copyright. Dit betekent dat het vrij distribueerbaar is, mits de source meegeleverd wordt. De source (een Intel assembler file) bevindt zich dan ook in alle distributies. U heeft hiermee de mogelijkheid fouten zelf te (laten) verwijderen. Als u een gemodificeerde versie verspreid, moet U dat ook weer doen onder het GPL copyright. Dit houdt in dat U de gemodificeerde source mee moet leveren. Het generieke systeem is ook vrij beschikbaar in broncode vorm. Dit werd gebruikt om de documentatie, source en testen te maken. Hiermee kan je ciforth aanpassen voor een andere operating systeem; het wordt aanbevolen indien men zelf ingrijpende wijzigingen aan wil brengen aan ciforth.

Copyright © 2000-2015 Albert van der Horst.

ciforth wordt beschikbaar gesteld onder de GNU Public License. Voor dit gratis beschikbare systeem aanvaardt de auteur geen aansprakelijkheid. Vragen, opmerkingen en probleemrapporten zijn altijd welkom.

Available ciforth compilers.

The newest version of ciforth is 5.4

64 bits, lina voor Intel Linux en Engelse documentatie in PDF en HTML. Also available as debian package.

32 bits, lina voor Intel Linux en Engelse documentatie in PDF en HTML. Also available as debian package.

64 bits, lina voor Arm Linux en Engelse documentatie in PDF en HTML

32 bits, lina voor Arm Linux en Engelse documentatie in PDF en HTML

64 bits, wina voor Window XP and later, PDF en HTML

32 bits, wina voor Window XP and later, PDF en HTML

Not yet available, in the works, but see below:
64 bits, xina voor Apple en Engelse documentatie in PDF en HTML
32 bits, xina voor Apple en Engelse documentatie in PDF en HTML

xina 5, 32-bits, ciforth voor Apple OSX en Engelse documentatie in PDF, PS en HTML (xina-5.2.1.tar.gz )

De lina for arm werkt op de RaspberryPi onder Raspbian linux, OrangePiOnePlus en NanoPiM4 and the  original Raspberry Pi .

Tested on Raspbian linux, i.a.

There are many boards, and we have obviously not tested all.

 Obsolete versions

 lina version 4 voor linux en documentatie in PDF, PS en HMTL (ca 1100 kb). 

wina 5.1, (dpmi,korte filenamen etc.) ciforth voor oude 32 bits Windows 95 t/m versie 7 en Engelse documentatie in PDF, PS en HTML

(wina5d1.zip ). Hier komen geen updates meer van.

Dit is gebaseerd op DPMI, dus bijvoorbeeld 8+3 filenamen. Hier komen geen updates meer van.

Hier komen geen updates meer van.

What is ciforth?

ciforth is a "common Intel" Forth, so a Forth that runs on all Intel systems.

Why Intel? This results from the way this Forth is programmed. Forth systems are programmed in assembler, in C, or in Forth itself. The systems you know from the Forth gg are all written in Forth, which is indicated by the theatrical name MetaForth. The old Forth (the one in which the program is written) and especially the metacompilation program sometimes lack in documentation, however good the new Forth may be. Forth's written in in C or assembler at least are based on external tools that are present in good quality. A Forth in C has also an advantage to be portable over all systems that have the same c-library . An example is gforth that runs on all Linux and many Windows systems.

An assembler Forth also has good portability, but an assembler program is by nature tied to a specific processor. ciforth is programmed in assembler, in this case the Intel. It can only be build for Intel based systems: Intel Linux, Windows and MSDOS. Their names are lina , wina and mina.

We know subroutine threaded, direct threaded and indirect threaded Forth's, in order of speed and reverse order of flexibility. ciforth is indirect threaded. This is related to an other meaning of ciforth: computer intelligence Forth. It must be a Forth that can understand itself and modify itself.

Properties

• ciforth has excellent documentation, and is professionally tested.
• ciforth consists of a relatively small kernel written in assembler, say one terminal screen worth of words. All other words are loaded as needed. To this end a handy library mechanism has been designed.
• ciforth has as starting points simplicity, multi-purpose and transparancy. Speed takes a back seat.
• ciforth is relatively slow, but it is easy to make a snapshot of the system, programs loaded included, to store and start again.
• ciforth can be used to make real programs. Not -- what you have to say about most Forths -- it can "also" be used to make programs.
• ciforth cooperates well with programs of Linux and Windows. In other words, it has some properties of an IDE (integrated development environment).

Documentation

Good documentation for a system with several variants is a problem that seldom is solved nicely. For example, because ciforth comes in 16 and 32 bits versions, you could have all over the documentation such confusing sentences as "two times the bit size of the system, see introduction". Or "this section only applies whenever files have an owner, like on Linux or Windows versions after Windows 95.".

The solutions that I have chosen, is to not write the documentation but generate it from a single source file (together with the tests).

So the source file cannot be assembled directly. It means that at the place where the code of for example the word DROP resides, we also find the explanation about the stack effect, and several tests. The documentation system collects these pieces of documentation, groups them and places them in the glossary in alphabetic order.

The code for TYPE differs for MSDOS and Linux, but the source file is common for all Forth's. The MSDOS TYPE is surrounded by

_HOSTED_MSDOS_({ HEADER({TYPE} ..... ... ... ... })

and the Linux TYPE code by

_LINUX_({ HEADER({TYPE} ..... ... ... ... })

In generating the assembler file for Linux the part for MSDOS is ignored. This is comparable with the use of [IF] [THEN] [ELSE] in Forth source, of #if #then #else in c-source. The juicy thing is that this is used at the same time to select documentation that is relevant to DOS c.q. Linux.

In other words, after the configuration step we have dedicated, albeit raw documentation, a file with tests and a file accepted by the assembler.

It will be clear that we need a lot of auxiliary programs. Unix is well equipped to do this kind of automatic text processing. The Unix used by me is LGX : Linux, GNU, X-windows. The system is also flexible in the sense that it can generate any output format, like PostScript and HTML. I hear you say, "I want Word". Helas that is not possible, not because Unix or I are not clever enough, but because the Word format is secret and/or you must pay just to use it. Windows help files similarly have a secret format. A same situation exists with PDF files, but less severe.

testing

ciforth is one of the few Forth I know, that fulfills professional demands with respect to testing. The testing is simple, in principle. The work consists of exhaustive tests for each Forth word. We use so called redirection, which means that we take care that the program gets it input from a file instead of the keyboard. This uses the symbols '<' en '>' . It amounts to saying that invoer.frt consists of all tests that we could have typed manually in principle. After

ciforth < invoer.frt > uitvoer

the file uitvoer contains everything that would otherwise have appeared on the screen. The file uitvoer is now compared to the expected output that is also generated automatically. If there are no difference, everything is all right. This is called a regression test, if it is run diligently after every change. Redirection works for MSDOS, Linux and Windows.

Real programs

With real programs I mean programs about which the user need not be aware that they are written in Forth.

Starting a Forth often is a nerve wrecking experience.

For example, if you want to use iForth for the tingle-tangle's you have to handle environment variable, library directories and start up commands, unless you also want to load the tingle software by hand, leave alone the installation of iForth. However all the above is necessary before you can say to some one: "There you go! Tingle along!"

It can be different. I have written a flash programmer for an Elektor board. It sits on the Desktop as an icon (KDE linux). In order to flash the device, you look up the binary with the file viewer and drop it on the icon. Only if there are problems, they are shown to you. Under MS-Windows during the flashing one would show a moving mill for reassurance. MS-Windowers are not used to reliable programs.

For real programs it is necessary that Forth start up with a specific word, instead of the OK-loop, and that we can pass information to the Forth, like in this example the file to be flashed. The first is called a "turnkey mechanism". The second is "passing arguments" We don;t want all this code in the kernel of the Forth. Therefore let us see where we want to store the code to be loaded.

The LAB library mechanism

Let us have a look at the words NIP and TUCK with stack effect ( a b -- b ) and ( a b -- b a b ). Those belong in the same category as SWAP and OVER. If some one presents you a Forth program, chances are that those words are used. They are ISO-standard, even if they are not in the CORE wordset. It is natural to have support for them. You can include them in the Forth kernel. That I don't like. Where is the end? An other solution is to put those two together in a small file niptuck.frt in a special directory. There is no end to this either, and the special directory presents a problem to the installation as well as the user, and the solutions are probably system-dependant. In MS-DOS you would use environment variables, in MS-Windows the registry.

The library mechanism such as used in ciforth has the effect that it is possible to load those two definitions while ciforth needs to know the where abouts of merely one file (the "lab" file). (In the section configuration we will see a system-independant solution for the configuring of ciforth.) In fact it is very simple. The file is subdivided in blocks of 16 lines. The first line of every block is an index line and contains in comment the names of the words that are defined in the block:

The stack comment is pitiful, but we don't want to present just yet the style of comment used in ciforth.

If we want to load a program that uses TUCK , we put the line

WANT TUCK

up front, or in an over-all load-file. Forth inspects all index-lines and loads the first block where the word TUCK is present in the index line. The fun starts for real, if you realize that in this block too WANT can be used. What if the definition of TUCK would comprise 5 blocks? Then we just put TUCK in the index-line of each block. Those blocks are loaded, it is only terminated once TUCK has been defined. The effect is that the 5 next blocks are compiled. It could be that a word needs to be coded differently for Windows and Linux. The library mechanism ignores code that is not applicable to the Operating Sytem at hand.

Note that orginally WANT was called REQUIRE , which is available on other Forth with a different meaning.

N.B. lab means: "library addressable by block". The blocks themselves are numbered. That can be handy for other things, for example the error messages are present from block 32, to the tune of 16 per block.

A drag and drop secret

Everybody knowS the old-fashioned way to start program's, like in MS-DOS

C:> XCOPY C:\FORTH D:\BACKUP\FORTH /S

This is called a command line. We see here the prompt, the program name, and three textual arguments. The textual arguments are separated by blanks, in Forth fashion. At least on Unix. In MS-DOS one can concatenate the /S to the previous argument. The third argument is special, a so called option. It doesn't tell the what -- which files are to be treated -- but the how. The normal way to write options is with a hyphen, so -s.

The secret of drag and drop is that command lines are not old fashioned at all, but are instrumental to the drag and drop mechanism. For example if you drop the file KNIPPER.BIN on the icon of my Elektor flash program (el-flash), the system in fact executes the following command:

el-flash KNIPPER.BIN

The remainder of the secret is the installing of the icon. It is possible to include fixed options, such that the effect is

el-flash -c2 KNIPPER.BIN

In this case it means that COM2 is where the flash programmer is connected. In order to make real program's, handling command lines gets us half way. The remainder is terribly system-dependant anyway.

Command lines and arguments

From the Forth perspective we know what we want, to wit the word ARG[]. If lina (ciforth for linux) has been started with

lina -b we gaan naar Rome

we want to be able to say

3 ARG[] TYPE

and then the third argument: "gaan" is to be printed. You know the drill by now:

WANT ARG[]

It may have been quite an effort, but the word ARG[] is present in the lab, one for Linux and one for MS-Windows/MS-DOS.

But ... how does Forth which to load? In fact it doesn't know, it just loads everything. In the block for Linux the word ?LI has been added and in the one for MS-DOS the word ?MS :

block 172: ( ARG[] get_argument_vector ) CF: ?LI \ AvdH : ARG[] .... block 173: ( ARG[] get_argument_vector ) CF: ?MS \ AvdH : ARG[] ....

CF: takes care that the words ?LI and ?MS are known. If it is not a linux system, loading stops after ?LI . Because ARG[] is still not known after loading block 172, the lab-mechanism keeps on searching and finds it in block 173.

P.S. -b just loads screen 2, but is is empty.

Saving and configuring the system

An essential facility for Forth is storing a configured Forth and restarting it in the same shape as when it was stored. This is called SAVE-SYSTEM in the usual systems. In many systems (such as iforth , gforth), this is too difficult and hampers usability of the Forth in question. So you want the following command's:

INCLUDE chess.frt S" chess" SAVE-SYSTEM

And then after typing "chess" just start playing chess.

It is of course useful not to have to reload a bunch of files, but there is more. Suppose you don't want to use the standard lab file, but instead

/home/albert/experiments/myforth.lab.

ciforth supplies command's for those needs.

"/home/albert/experiments/myforth.lab" BLOCK-FILE $! BLOCK-EXIT R/W BLOCK-INIT

I show those command's , not to explain them, but to make clear that one doesn't want to type them again and again. This kind of configuration data too is stored inside the saved system. The "burning into" an executable program of configuration data replaces the use of the registry in Windows and the so called environment variables in Linux. The nice thing is that is is system dependant. There is a hitch, virus detectors may hate it.

Making a program

Once you can store a modified system and restart it, you can make a regular program. Instead of the OK-loop, after starting up, some other required Forth word must run. This is trivial indeed. "somewhere" that word is filled in. In Forth circles the pompous word TURN-KEY is used. I just call it compiling a program. It is made easier in ciforth with the -c option

lina -c aap.frt

This compiles the file aap.frt, makes the file aap, (on MS-operating systems an AAP.EXE ) and takes care that aap starts with the last word defined in aap.frt. After that word is ready, BYE is executed automatically. Of course in aap.frt you can use the word ARG[], such that we have everything that is needed to make real program's.