Book Making Q & A

Hi Kevin:

First of all a big thanks for the new site. Its an excellent source of advice which I have recommended to several freinds and colleagues and on chess forums.

I've been into engine chess for about 2 years but have stilllearned valuable tips from your advice on book making and particularly book tuning. I read your advice on book making just now and it hadn't occurred to me to select a small sample as a "learn from " base. I have always done the learn from Database against the whole game base, whic may explain some anomalies I have been seeing in my engine play !

I have been using the Tour Book 14.0v with some success ove rthe past couple of weeks and have just seen that you have released version 14.Oz.

Two questios: What source have you used for the book. Is it all Rybka games or a mixture of engines ?

Is it all blitz games or some longer ones mixed in ? What you are changing between versions ? I can see that the size of the book in terms of number of games is still N=18,497 but the ctg file size is increasing each time ?

Many thanks,

Mark

_______________________________________________________________________________________________

Mark:

Thank you very much for your kind words.

Yes, I have found that in general the learn base should be considerably smaller than the main book base. The main base must contain many obscure and mostly defensive lines against a variety of threats. While the learn file should consist of the strongest current opening lines for both black and white. Tour Book 14.0z is primarily a blitz book tuned for 3-0 time controls. However it does quit well at 1-0 and 5-0 times. At 16-0 it will begin to show some weakness. It is made from games that are mostly 3-0 and 5-0 times.

It is my thinking that engine opening books should be made time and engine specific. I brake the time controls for books down like this; one book for 3-0 one for 16-0 and one for 30+ times. Also as I mention different engines favor different positional themes you should use a book that is tuned to the engine that you are playing.

The reason the number of game in the book is staying the same and the configuration file is increasing in size is that the book is in the hand tuning phase. All new lines and moves are being added by hand after deep positional analyses.

Best Regards:

Kevin:

Book Making Q & A II

Hi Kevin,

Hope you don't mind but could I just run one last thing by you.

As an experiment yesterday I followed your book making advice to make a seed book and started with all server games from January 2007 to January 2008. I then removed all draws and games with less than 35 moves and from what was left I extracted all games where elo of both players was at least 2700. This gave a game base of 31,605.

I then created a learn base using games of Big Rabbit, BigMc, Zapchessbook and Golden Boy (these seem to be consistently high elo players) giving me a learn base of 3,667. I then compiled the book using Absolute 100 (I meant to ask - do you use Absolute or ECO Relative to compile book ?). I then ran a learn from DB using my learn base ticking black and white wins.

I had short run on the server using the GUI book settings you suggested and when I looked back at some of the losses I found that my book played quite long into what was not a very good line. When I checked against Tour Book 14.Ov and 14.Oz I found that your books had stopped well short of that position eg. stopped at move 14 whereas my book line played on to move 23. Is this because as part of tuning you have cut short certain lines and removed them , or is maybe because you compiled using ECO Relative rather than Absolute ?

Regards,

Mark

------------------------------------------------------------------------------------------

Mark:

It appears that you have done everything right. As for absolute or ECO, I always use Absolute as it will give the book better depth.

As for the new book following unproductive lines to great depths, yes this will happen at first. As you play the book more and add games that you have won as well as your opponents wins against your book; the new lines will begin to supersede the losing lines. At this point you have not set any red or green moves to be followed, so with the recommended book settings given; the fledgling book is using only the percentage of games won with any given move as it criteria for choosing.

As games are added to the learn file you will almost always notice the depth of the book reducing. (always add the games that you wish your book to learn, to both the main game base and the learn file) The slow reduction of the books depth in this stage of development is a good thing. After you have added 2000 or so games (adding primarily games played with the book you are adding them into) the book will stabilize. Once this happens you can go to the hand tuning phase and it is here that you can extend the winning lines to their full depth.

Good luck and Best Regards:

Kevin:

Book Making Q &A III

Kevin:

I wanted to ask you a question regarding book building...

What are the pro and con of building a book out of 2 or more books?

Thanks!

Anonymous:

-----------------------------------------------------------------------------------------

Anonymous:

This is some thing that I do not have a lot of experience with. You are I believe referring to the Fritz GUI function in Books…Import Book.

“Import book” merges the contents of a different book into the current one.

“Import old book” converts a book in the older (FBK) format and imports the contents.

The program will also convert and import books from the programs Genius, Rebel and Chessmaster. If a move is already in the openings book it will be marked as a preferred tournament move.

I put this function in much the same category as the GUI Book Learn. It works on an elementary level but probably will not produce the results you are looking for. The problem is that when you use this function to attempt to merge two all ready highly advanced books. (books with extensive move weights and hand enabled red-green tuning marks) The weights and move preference marks become convoluted creating conflicts for the GUI when the book is in play. It would be nice if the best lines of two books could be combined so easily. It also would nice if book learning actually worked.

However having said this there are techniques that some book makers use to utilizes the book learn function. Playing a newly created book without weights or marks with the book learn turned on to identify faulty lines; that can then be later addressed and altered manually.

I am sure that some book development technique may successfully employ the Import Book function but it not something that I have ever used.

Kevin:

Book Making Q & A IV

Hi Kevin,

I will be attentive about your updated books. I have an old and poor pc (AMD 2.4 Ghz Athlon xp) with 768 ram. Nevertheless I win some dual core 2 with 4 threads and draw with some 8 threads.

I am very happy to have found a good book. only a question I need to know :

I played with 72 megas of hash tables and 4 megas of table bases. Is this configuration correct? what will be the best configuration with your book?

Jean:

------------------------------------------------------------------------------------------

Jean:

The amount of hash table file size that you use is dependent upon your system and the engine being used. The conventional wisdom is to use one half of your available system RAM. I think I agree with this in long time control games. However in blitz play 3-0 5-0 times, you do not need that much. A smaller hash size (with most engines) will give you a bit more nodes per second but will reduces the depth in Ply especially in the end game.

So it is a compromise and you must find the setting that gives you equipment the best performances. Also most, if not all engines will only use hash settings in multiples of 8 (16-32-64-128 256-512) so your 72Mb is being rounded down to 64Mb.

When you are trying to decide the optimum setting look closely at the depth in the end game play. If your engine does not seem to be getting good depth here you may need to go to the next hash size.

As you have described your system I would say that you have it about right… 64 or 128 in blitz play.

As for the EGTB Cache this depends on how many egtbs that you are using. The speed of your hard drive can come into play with egtb as well.

This Cache is actually only the references file for the bases stored on the hard drive. The RAM stored file is interrogated by the engine as the number of pieces on the board become few. By the way it may take several games for this file to load up fully into the cache.

This is what I have found to be about right:

For all 3-4-5 man about 5 Giga bites…use 8Mb of cache

For all 3-4-5 and a few 6 man (CB Turbo 3) 48Gb…use 16Mb of cache

For blitz play I use 140Gb of EGTBs all on a Raptor 10,000rpm hard drive, with a cache size of 32Mb.

Kevin:

Book Making Q & A V

Hi Kevin:

Just came across your web page which I found very interesting. Regarding your settings for the book, can you clarify please.

Book settings:(Frayer Tour Book)
Use Book: on
Tournament Book: on
Varity of Play: max - (all the way to left)
Influence of learn Value: max + (all the way to right)
Learning Strength: max - (all the way to left)
Minimum games: 1
Up to move: 100

On my Fritz 11 the max settings are all the way to the right and the min settings all the way to the left for all three settings not just learn value..

Regards:

Mike:

-----------------------------------------------------------------------------------------

Mike:

You are right it is not as clear as it could be. Varity of Play is Max - (Minus) or the minimum variety setting. You must look close but two are max - and one is max+.

My intent is for the book to use the minimum amount of variety pf play and learn strength and the maximum amount of influence of learn value. (all the way; left, right, left)

I want the book to play a narrow repertoire of the sharpest lines. (especially in blitz times) I do not want the book to use the GUI learn function as I feel it is detrimental in such a narrowly focused book. The learn function does not learn any new lines or add any moves to you book at all. What it does is to mark moves as being successful or unproductive. In order for the book to actually learn, it needs a little assistance from us using the book development techniques and hand tuning process I have talked about in my blog.

Hope this has been of some help, please do not hesitate to write me any time with questions.

Kevin: