Another Attempt At Programming

This is a project I started years ago while I was at collage using Delphi Developer 2.0. I think I was doing fairly well. Although I was having trouble with the sheer amount of data this project can generate.

So what is it? Well it's a program intended to narrow the odds of winning the lottery by eliminating combinations of numbers already drawn and using the historical data to predict the next draw.

The last time I attempted this I tried to do it all as one big program. This time I'm breaking it down into little bits. One program for each element that does one thing and does it well.

The first task is to generate all possible combinations of numbers, ordered sequentially. It might turn out I don't really need this part. But it gets me started. I've kept a diary of my progress so far. Which will be published in this post. I think I'll also start a blog specifically for this project.

Project	Lotto Predictor
Module Name	lotto-sng-sqldatabase.py lotto-sng-txt.py
Author	Kevin Lynch
Created	17.03.2013
Brief	Write a program that generates all sequential combinations of numbers in a given range. The program must save the resulting output to an sql database.

17.03.2013: Project Established

“lotto-sng-sqldatabase.py” is a program designed to generate every possible sequential number combination for a given Camelot lottery draw game. The games primarily being targeted are the Lotto, Euro Millions and Thunderball.

The draw lines generated will be saved to an SQL database for later analysis by a different module.

Standard modules being imported include;

• os – helps with hos OS functions.
• sys – helps with host os functions.
• termios – helps with host os terminal functions.
• fcntl - helps with host os terminal functions (I think … not entirely sure).
• struct – helps with data structure functions.
• string – helps with string manipulation functions.
• apsw – SQL wrapper for Python.

Each game type will be defined as a class in it's own right based on a generic class. This should help to avoid duplication of code while at the same time allowing game-specific modifications.

• class rootg(): – is the generic game object.
• class lotto(): – is the Lotto game object.
• class eurom(): – is the EuroMillions game object.
• class thund(): – is the Thunderball game object.

All game specific objects need to pass game specific data to rootg objects. Specifically the number of main sequence numbers, special numbers (bonus ball, thunder ball, lucky stars) and the upper and lower limitations on these number groups.

Phase 1 in developing this module will be to simply get the program to generate the number sequences and print them to the terminal window.

Phase 2 will be the development of the sql database and directing output to the database.

Phase 3 will be a final polish and will not be considered essential.

17.03.2013: Basic Structure Established

The four main classes have been established as stubs along with the main “while” loop and a stub function for the main menu. Three other stubs were added. “do_lotto”, “do_eurom” and “do_thund”. These are intended for program flow control. However they will likely be removed. I think the class structure will probably provide enough flow control.

A final exit message was also added primarily as a test for the main “while” loop. Don't judge me, I'm paranoid!

17.03.2013: SNG Code Prototyped

SNG code has been prototyped as a function. The next step is to convert it to a class object and optimise it as a generic object so that the same code can be utilised for all targeted games.

It may also be necessary to periodically save generated sequences to file. The Python list object currently takes up 800MiB of RAM and counting with the third column at 31. It's maximum is 46.

17.03.2013: Proper Code Name Being Considered

Toutatis, the Celtic god of protection, war and wealth is being considered as a code name for the project. It needs a proper code name.
 

18.03.2013: Duplication Checking Added

Tests added to check for duplicate lines and numbers. Currently suppresses output. May need to do this the long hard way.

18.03.2013: Duplication Checking Added Again

A simple way to remove duplicates found. But it has an odd effect on main ball 6. It prevents it from reaching it's maximum of 49 and duplicates every line when main ball 6 has a value of 48.

18.03.2013: Reworked Number Line Generation And Duplication Removal

Method of generating number lines reworked with some basic duplication removal. Some anomalies still show up. It might better to remove these when the final list has been produced. Although this list will be over 2GiB in size.

I'm now using a single “while” loop were I was using nested “for” loops before. The code is much cleaner, more compact and should be adaptable to a generic game object. The reworked code is also exceptionally fast, producing hundreds of thousands of combinations in seconds.

Two small utility functions were also created. “inc(v,line)” and “dec(v)”. “inc(v,line)” increments “v” and checks for it's existence in line. It continues until if finds a value for “v” not already in “line”.

“dec(v)” subtracts 1 from v.

20.03.2013: While Does Not Stop

The “while” loop used to generate number combinations does not stop at the predicted final combination. Meaning the stop condition test has failed. This will need to be reworked again.

Upper limit enforcement will also need to be implemented. Ball 6 is some how incrementing to 50.

20.03.2013: The Bonus Ball Is The Root Of All Evil

It turns out generating the main line numbers and the bonus ball in one step is the root of all the problems so far. Removing the bonus ball from the equation allows the stop test condition to kick in and stop the “while” loop.

There's really no particular need to generate the bonus ball. So I will be ditching that for now.

Using “if v not in line:” is still allowing the odd anomalous result. I think I will be ditching “not in” in favour of writing my own function.

20.03.2013: Duplication Issues Resolved!

Resolving duplication of individual values and entire lines has been an issue throughout the project. I'm now confident this problem has been resolved. I have done this by wring my own function to replace “not in” and by ensuring the value in the column to the right is always greater than the value in the column to the left.

I believe this will prevent duplicate lines and values while still generating the full range of unique combinations that would be valid in the UK National Lottery.

20.03.2013: Change Of Plan

The original plan was to have one program that could do everything. I now think it would be better to create smaller utility programs to do the one-off tasks. The main program can always call these utility programs if need be. This should simplify development.

The program has been cleaned up to reflect this change of plans. Using Python print formatting and the tee utility at the terminal, output can now be saved to a comma delimited file. However the final utility will still likely build an SQL database.

20.03.2013: Return Of The Failure To Stop Bug

The program has suddenly started failing to stop again. I can't figure out why. Balls 2,3,4,5 and 6 burst their limits by 1.

20.03.2013: Stop Bug Resolved … Again!

Problem resolved again by adding extra checking to “inc”.

21.03.2013: First Perfect Set Of Results

I've just checked the results this morning and they appear to be perfect. The program stopped when it should have and there don't seem to be any anomalous results. The final file is 152 MiB in size. This is a little worrying as I'm fairly certain I used to get a file GiB in size with Delphi. However at first glance everything seems to be present and correct.

The next step is to write a program that can load two text files and compare the contents. When a matching line is found it will be marked “VOID”.

22.03.2013: Working With Text Files

I've decided to stick to working with text files for the time being. Writing small programs that do one thing and one thing only seems to be the best way forward at the moment until I get a better grasp of programming with Python. So with that in mind “lotto-sng-sqldatabase.py” will become “lotto-sng-txt.py”.

This program does not generate the final text file, but rather generates text output to the terminal. The output must be redirected to a text file using piping or bifurcation. Which is virtually universally supported in Linux distributions. Mac and Windows terminals also support output redirection. Not that these platforms will be tested. They're not a priority.

22.03.2013: “lotto-sng-txt.py” Usage

Simple output to terminal command:

./lotto-sng-txt.py

Redirected output command:

./lotto-sng-txt.py > whatever-file-name-you-like

Bifurcated output command:

./lotto-sng-txt.py | tee whatever-file-name-you-like

22.03.2013: Final Code Listing For “lotto-sng-txt.py”

 

#!/usr/bin/env python

# Project : Lotto Predictor

# Module Name : lotto-sng-txt.py

# Author : Kevin Lynch

# Created : 17.03.2013

# Brief: Write a program that generates all sequential combinations of numbers in a given range.

# Function Definitions:

def inc(v,line,m):

    go = True

    while go == True:

       if v < m:

          v = v + 1

             if v != line[1]:

                if v != line[2]:

                   if v != line[3]:

                      if v != line[4]:

                         if v != line[5]:

                            if v != line[6]:

                               go = False

       else:

          return v

    return v

def do_lotto():

    god = [True,[1,44],[2,45],[3,46],[4,47],[5,48],[6,49]]

    vline = ["*",1,2,3,4,5,6]

    print "%02d,%02d,%02d,%02d,%02d,%02d" % (vline[1],vline[2],vline[3],vline[4],vline[5],vline[6])

    while god[0] == True:

       if vline != ["*",44,45,46,47,48,49]:

          if vline[6] < god[6][1]:

             vline[6] = inc(vline[6],vline,god[6][1])

          elif vline[5] < god[5][1]:

             vline[5] = inc(vline[5],vline,god[5][1])

             vline[6] = inc(vline[5],vline,god[6][1])

          elif vline[4] < god[4][1]:

             vline[4] = inc(vline[4],vline,god[4][1])

             vline[5] = inc(vline[4],vline,god[5][1])

          elif vline[3] < god[3][1]:

             vline[3] = inc(vline[3],vline,god[3][1])

            vline[4] = inc(vline[3],vline,god[4][1])

          elif vline[2] < god[2][1]:

             vline[2] = inc(vline[2],vline,god[2][1])

             vline[3] = inc(vline[2],vline,god[3][1])

          elif vline[1] < god[1][1]:

             vline[1] = inc(vline[1],vline,god[1][1])

             vline[2] = inc(vline[1],vline,god[2][1])

          print "%02d,%02d,%02d,%02d,%02d,%02d" % (vline[1],vline[2],vline[3],vline[4],vline[5],vline[6])

          else:

             god[0] = False

# Main Program:

    do_lotto()

So there it is. The first component is way simpler than I first imagined and planned it to be. But with the tee command available to create the text file for me I see no reason to duplicate this function at this early stage. Now I just need to do the elimination part and the prediction part. Which will be really easy. ... :(

Note to Blogger developers. Is there any real reason why Blogger can't retain tabbed indentation?

Programming In Python 1: The Cookbook Project

Finally it's finished! Well as finished as it'll ever be. Almost a year a go I started learning Python, the programming language on and off. Which really isn't an ideal way to learn.

As with anything we want to learn in life I needed source materials, examples and exercises to complete. I found these in the Ubuntu orientated on-line magazine Full Circle. One of the early tutorials dives straight into writing a simple program that creates, loads and updates a database of cooking recipes.

The data in the database in this exercise isn't particularly important. It's the lessons learnt in writing the program that creates and manipulates the database. And fortunately Python is a very rich language with a lot of additional modules that can be called upon to enhance the power of software written in Python.

Personally I don't think what I've produced is particularly impressive. But it's where I'm at. For example I really should be making better use of Python's class objects. I guess I still have some habits from learning COMAL and Turbo Pascal bouncing around in my brain getting in the way. I guess I can only get better right?

With some basic concepts learnt, the next challenge is to create a program that will win me the lottery. On the off chance anybody is interested I've included my source code below.

#!/usr/bin/env python
#------------------------------------------------------------------------------
#
#    Program Title    :    Cookbook Database
#    Local Filename    :    sql-0004-cookbook_database.py
#    Author            :    Kevin Lynch
#    Created            :    04.06.2012
#
#------------------------------------------------------------------------------
#
#    Brief:    Write a program capable of storing recipes for later retrival.
#
#                * The program should be menu driven.
#                * Include a search function allowing users to search by;
#                    + Recipe title,
#                    + Author,
#                    + Ingredients.
#
#                * The program should also be capable of creating new cookbooks.
#                * Adding new recipe entries to each relevant book on demand.
#                * Removing unwanted entries from a book on demand.
#                * Program output and interaction prompts must be presentable.
#
#------------------------------------------------------------------------------
#
#    Additional Credits:
#
#        This project is based on the Python tutorials published by
#        Full Circle Magazine. So far as this author can tell the original
#        "cookbook" tutorial was written by Greg Walters of
#        "RainyDay Solutions, LLC" and "www.thedesignatedgeek.com".
#
#        Terminal dimensions code courtisy of Grant Edwards.
#        http://bytes.com/topic/python/answers/607757-getting-terminal-display-size
#
#------------------------------------------------------------------------------
#--|    Import Modules    |----------------------------------------------------

import os
import apsw # SQLite wrapper.
import string
#import webbrowser
import termios, fcntl, struct, sys

#------------------------------------------------------------------------------
#--|    Class/Object Definition Section    |-----------------------------------
class aScreen():
    def __init__(self,aTitle,aBorder,aJustify,aMessage,aContent,aOptionlist):
#        Get Dimentions
        s = struct.pack("HHHH", 0, 0, 0, 0)
        fd_stdout = sys.stdout.fileno()
        x = fcntl.ioctl(fd_stdout, termios.TIOCGWINSZ, s)
#        print '(rows, cols, x pixels, y pixels) =',
#        print struct.unpack("HHHH", x)
#        return struct.unpack("HHHH", x)
        s = struct.unpack("HHHH", x)

        self.iDefault = 'Press "Q" to quit <::> '

        self.H = (s[0] - 6)
        self.W = s[1]
        self.J = aJustify # Tells the aScreen object which type of justification to use.
        self.T = aTitle
        self.B = (aBorder * self.W)
        self.P = 1
        self.M = aMessage # Should be a short single line instruction.
        self.C = aContent # Should be a list. Each list entry will correspond to a line in the terminal.
        self.O = aOptionlist # List of valid responses for this screen.

    def checkLine(line):
        pass

    def formatScreen(self):    # Formats the screen output. Works for showScreen() and should not be called directly.
        # Initialise screen segmentation.
        seg1 = [self.T,self.B,' ']
        seg2 = []
        seg3 = [' ',self.B]
        lines = []

        # Copy charcaters one at a time from self.C to creat a string of a maximum length of self.W - self.P
        if len(self.C) < self.H:
            j = len(self.C)
        else:
            j = self.H

        for i in range(0,j):
            line = self.C[i]
            if len(line) < self.W:
                seg2.append(' %s' % line)
            else:
                j = (self.W - self.P)
                while len(line) >= self.W:
                    if line[j] == ' ':
                        seg2.append(' %s' % line[0:j])   
                        line = line[(j + 1):len(line)]
                        i += 1
                    else:
                        j -= 1
                seg2.append(' %s' % line)
                i += 1

        # Add filler lines.
        if len(seg2) < self.H:
            for i in range(len(seg2),self.H):
                seg2.append(' ')

        # Add everything to one big list.
        for i in range(0,len(seg1)):
            lines.append(seg1[i])
        for i in range(0,len(seg2)):
            lines.append(seg2[i])
        for i in range(0,len(seg3)):
            lines.append(seg3[i])
        return lines

    def errorScreen(self):
        pass

    def showScreen(self): # Displays the current screen.
        loop = True
        lines = self.formatScreen()
        while loop == True:
            # Print the title, main body and borders of the screen.
            for i in range(0,len(lines)):
                print lines[i]
            # Prompt the user for input from the keyboard and verify the response.
            kbd = raw_input(self.M)
            for i in range(0,len(self.O)):
                if kbd == self.O[i]:
                    loop = False
                elif self.O[0] == 'pass':
                    loop = False
                else:
                    self.errorScreen() # When an invalid option is made the user is told.
        return kbd

class newRecord():
    def __init__(self): # Initialise the new recipe class.
        # Variables for Recipes table.
        self.name = ''
        self.servings = 0
        self.source = ''
        # Variables for Instructions table.
        self.instructions = ''
        # Variables for Ingredients table.
        self.ingredients = []
        # General variables needed for this record.
        self.recID = 0

class newDB():
    def __init__(self,dbname): # Initialise the Cookbook class.
        global connection
        global cursor
        self.totalcount = 0
        connection = apsw.Connection(dbname)
#        connection = apsw.Connection("cookbook1.db3")
        cursor = connection.cursor()

    def addRec(self,rec):
        sql = 'INSERT INTO Recipes (name,servings,source) VALUES ("%s",%s,"%s")' % (rec.name,str(rec.servings),rec.source)
        cursor.execute(sql)
        sql = "SELECT last_insert_rowid()"
        cursor.execute(sql)
        for x in cursor.execute(sql):
            rec.recID = x[0]
        sql = 'INSERT INTO Instructions (recipeID,instructions) VALUES (%s,"%s")' % (rec.recID,rec.instructions)
        cursor.execute(sql)
        for x in range(0,(len(rec.ingredients) - 1)):
            sql = 'INSERT INTO Ingredients (recipeID,ingredients) VALUES (%s,"%s")' % (rec.recID,rec.ingredients[x])
            cursor.execute(sql)

    def deleteRec(self,rid):
        sql = "DELETE FROM Recipes WHERE pkID = %s" % rid
        cursor.execute(sql)
        sql = "DELETE FROM Instructions WHERE recipeID = %s" % rid
        cursor.execute(sql)
        sql = "DELETE FROM Ingredients WHERE recipeID = %s" % rid
        cursor.execute(sql)
       
    def listAll(self): # Create a list of all recipes.
        res = ['%s %s %s %s' % ('Item'.rjust(6),'Name'.ljust(30),'Serves'.ljust(7),'Source'.ljust(30))]
        for x in cursor.execute('SELECT * FROM Recipes'):
            res.append('%s %s %s %s' % (str(x[0]).rjust(6),x[1].ljust(30),x[2].ljust(7),x[3].ljust(30)))
        return res

    def listOne(self,rec):
        sql = 'SELECT * FROM Recipes WHERE pkID = %d' % rec.recID
        for x in cursor.execute(sql):
            rec.recID = x[0]
            rec.name = x[1]
            rec.servings = x[2]
            rec.source = x[3]
        sql = 'SELECT * FROM Ingredients WHERE RecipeID = %s' % rec.recID
        for x in cursor.execute(sql):
            rec.ingredients.append(x[1])
        sql = 'SELECT * FROM Instructions WHERE RecipeID = %s' % rec.recID
        for x in cursor.execute(sql):
            rec.instructions = x[1]
        return rec

    def searchDB(self,sql,option):
        try:
            if option != '3':
                # Do search for options 1 and 2
                res = ['%s %s %s %s' % ('Item'.ljust(5),'Name'.ljust(30),'Serves'.ljust(6),'Source'.ljust(30))]
                for x in cursor.execute(sql):
                    res.append('%s %s %s %s' % (str(x[0]).rjust(5),x[1].ljust(30),x[3].ljust(20),x[2].ljust(30)))                   
            else:
                # Do search for option 3
                res = ['%s %s %s %s %s' % ('Item'.rjust(5),'Name'.ljust(30),'Serves'.ljust(6),'Source'.ljust(25),'Ingredient'.ljust(50))]
                for x in cursor.execute(sql):
                    res.append('%s %s %s %s %s' % (str(x[0]).rjust(5),x[1].ljust(30),x[2].ljust(6),x[3].ljust(25),x[4].ljust(50)))
        except:
            # Catch exception.
            res ['I have encountered a problem performing your search request!']

        return res
#------------------------------------------------------------------------------
#--|    Function Definition Section    |---------------------------------------
def LAR(db,title,message): # List all recipes in the database.
    c = db.listAll()
    lars = aScreen(title,'*','',message,c,['pass'])
    kbd = lars.showScreen()
    return kbd

def SAR(db,kbd): #Show a single recipe.
    # Show list of recipes to select from.
    if kbd == 'pass':
        kbd = LAR(db,'The Cookbook Project > Select A Recipe','Press "Q" to quit or make a selection <: br="br">    # Retrieve the choosen recipe from the database.
    elif kbd.isdigit() == True:
        rec = newRecord()
        rec.recID = int(kbd)
        rec = db.listOne(rec)
        # Display the results.
        c = [rec.name,' ','Preperation Steps:',rec.instructions,' ','Ingredients:']
        for x in range(0,len(rec.ingredients)):
            c.append(rec.ingredients[x])
        c.append(' ')
        c.append('Serves: %s' % str(rec.servings))
        c.append(' ')
        c.append('Written by %s' % rec.source)
        sars = aScreen('The Cookbook Project','*','','Press any key to continue <: br="br" c="c" pass="pass">        kbd = sars.showScreen()
    elif kbd == 'delete':
        kbd = LAR(db,'The Cookbook Project > Select A Recipe For DELETION!','Press "Q" to quit or make a selection <: br="br">        return kbd

def SRD(db): # Search for a recipe.
    # Determine search criteria.
    SRDS = aScreen('Search Database','*','pass','Press "Q" to quit or make a selection :> ',['Search by ...','1 - Recipe Name','2 - Author','3 - Ingredients'],['1','2','3','Q'])
    SRDSa = aScreen('Search Database By Recipe Name','*','pass',':> ',['What is the name of the recipie you would like to search for?'],['pass'])   
    SRDSb = aScreen('Search Database By Author','*','pass',':> ',['Who would you like to search for?'],['pass'])   
    SRDSc = aScreen('Search Database By Ingredients','*','pass',':> ',['Which ingredients would you like to search for?'],['pass'])   
    kbd = SRDS.showScreen()
    rec = newRecord()
    if kbd != 'Q':
        if kbd != '3':
            if kbd == '1':
                kbd = SRDSa.showScreen()
                sql = "SELECT pkID,name,source,servings FROM Recipes WHERE name LIKE '%%%s%%'" % kbd
                res = db.searchDB(sql,'pass')
            else:
                kbd = SRDSa.showScreen()
                sql = "SELECT pkID,name,source,servings FROM Recipes WHERE source LIKE '%%%s%%'" % kbd
                res = db.searchDB(sql,'pass')
        else:
            kbd = SRDSa.showScreen()
            sql = "SELECT r.pkID,r.name,r.servings,r.source,i.ingredients FROM Recipes r LEFT JOIN Ingredients i ON (r.pkID == i.recipeID) WHERE i.ingredients LIKE '%%%s%%' GROUP BY r.pkID" % kbd
            res = db.searchDB(sql,'3')

        SRDS = aScreen('Search Results','*','pass','Press "Q" to quit or make a selection :> ',res,['pass'])
        kbd = SRDS.showScreen()
        SAR(db,kbd)

def ARD(db): #Add a recipe to the database.
    nr = newRecord()
    screen = aScreen('Add A New Recipe','*','pass','Please enter the title of your recipe or press "Q" to quit :> ',[''],['pass'])
    nr.name = screen.showScreen()
    screen = aScreen('Add A New Recipe','*','pass','Please enter how many your recipe serves or press "Q" to quit :> ',[nr.name],['pass'])
    nr.servings = screen.showScreen()
    screen = aScreen('Add A New Recipe','*','pass','Please enter the name of the author of your recipe or press "Q" to quit :> ',[nr.name,'Serves %s' % nr.servings],['pass'])
    nr.source = screen.showScreen()
    screen = aScreen('Add A New Recipe','*','pass','Please enter the ingredients for your recipe or press "N" to move on :> ',[nr.name,'Serves %s' % nr.servings,'Written by %s' % nr.source],['pass'])
    kbd = screen.showScreen()
    loop = True
    while loop == True:
        if kbd == 'N':
            loop = False
        else:
            nr.ingredients.append(kbd)
            screen = aScreen('Add A New Recipe','*','pass','Please enter the ingredients for your recipe or press "N" to move on :> ',[nr.name,'Serves %s' % nr.servings,'Written by %s' % nr.source,' ','Ingredients'] + nr.ingredients,['pass'])
            kbd = screen.showScreen()
    screen = aScreen('Add A New Recipe','*','pass','Please provide instructions for prepairing your recipe or press "Q" to quit :> ',[nr.name,'Serves %s' % nr.servings,'Written by %s' % nr.source,' ','Ingredients'] + nr.ingredients,['pass'])
    nr.instructions = screen.showScreen()
    screen = aScreen('Add A New Recipe','*','pass','Do you wish to save this recipe? Press "Y" for YES and "N" for NO :> ',[nr.name,'Serves %s' % nr.servings,'Written by %s' % nr.source,' ','Ingredients'] + nr.ingredients + [' ','Preperation Instructions',nr.instructions],['pass'])
    kbd = screen.showScreen()
    # Now write all data to the database
    if kbd == 'Y':
        db.addRec(nr)

def DRD(db):
    screen = aScreen('Add A New Recipe','*','pass','Do you wish to continue? Press "Y" for YES and "N" for NO :> ',['WARNING!!! This section is for deleting recipes from your data base. This operation cannot be undone!'],['pass'])
    kbd = screen.showScreen()
    if kbd == 'Y':
        kbd = SAR(db,'delete')
        if kbd.isdigit() == True:
            db.deleteRec(kbd)

def mainLoop(dbname):
    db = newDB(dbname)
    t = 'The Cook Book Project > Main Menu'
    m = 'Press "Q" to quit or make a selection <: br="br">    o = ['Q','A','B','C','D','E']
    c = ['A - List All Recipes','B - Select A Recipe','C - Search Recipe Database',' ','D - "ADD" A New Recipe','E - "DELETE" A Recipe']
    mLscreen = aScreen(t,'*','pass',m,c,o)
    loop = True
    while loop == True: # Show the main menu until a valid option is selected.
        kbd = mLscreen.showScreen()
        if kbd == 'Q':
            loop = False
        elif kbd == 'A':
            # List all the recipes in the database.
            kbd = LAR(db,'The Cookbook Project > List All Records','Press any key to continue <: br="br">        elif kbd == 'B':
            # Select a recpie.
            SAR(db,'pass')
        elif kbd == 'C':
            # Search the data base.
            SRD(db)
        elif kbd == 'D':
            # Add a recipe.
            ARD(db)
        elif kbd == 'E':
            # Delete a recipe.
            DRD(db)
#------------------------------------------------------------------------------
#--|    Main Program    |------------------------------------------------------
# Title screen. Database file name will be asked for here.
intro = ['Welcome To The Cookbook Project','please enter the file name of your cookbook.']
title = aScreen('The Cookbook Project','*','pass','Press "Q" to quit or enter a file name <: br="br" intro="intro" pass="pass">dbname = title.showScreen()
# Main program loop.
if dbname != 'Q':
    if dbname == '':
        dbname ='cookbook1.db3' # This is a stub.
        mainLoop(dbname)
    else:
        mainLoop(dbname)

# Credits screen.
intro = ['The Cookbook Project Software.','Written by Kevin Lynch',' ','Original tutorial published by Full Circle Magazine.','Original code written by Greg Walters','Rainyday Solutions, LLC,','www.thedesignatedgeek.com',' ','Terminal dimentions detection by Grant Edwards']
title = aScreen(' ','*','centered','Press any key to quit <: br="br" intro="intro" pass="pass">dbname = title.showScreen()

Ubuntu Tip: Using Quickly To Build Applications - Basic Commands

What is Quickly?
Basically it's a command line based tool for building applications. So go ahead and open a terminal. Now it will be useful if you're familiar with the Python programming language. If you're not then there are tones of on-line resources to help you learn.

How to create a new application:
Type: quickly create ubuntu-application mybrowser

Quickly will create a directory with the same name as your application containing all it's files.

How to edit your application:
In the applications directory type: quickly edit

How to edit the GUI:
In the applications directory type: quickly design

How to run your application:
In the applications directory type: quickly run

How to package your application:
In the applications directory type: quickly package

You might also want to watch this video. It will take you through the development of a very basic web browser application.

Linux Is Not A Viable OS

Pageviews by OS 7 Jan 2012 - 5 Feb 2012

I just came across two comments in an idea on Dell's IdeaStorm claiming Linux is not a viable OS or some nonsense. Frankly I was gob-smacked to learn there are still some people promoting this FUD. So much so in fact I was moved to write quite a lengthy rant and then repost it here. Frankly if this stuff is still doing the rounds and people are actually believing it then we're not doing enough to get the message out that Linux currently dominates the OS market in it's many forms. It has a presence in virtually every device market pigeon hole. And it's the top contender in most. If you are a Linux user and Dell customer using Linux on Dell hardware then get your arse on IdeaStorm and tell Dell you want Linux!

That was the short version. Below is the long version I posted on IdeaStorm.

Linux is not a serious contender in the OS market? This is the sort of FUD and ignorance that keeps consumers away from Linux. Linux is very much a contender in the OS market. Which is why Microsoft lists desktop Linux as a threat to it's desktop Windows in it's tax returns. When netbooks first appeared Linux was such a massive threat to Microsoft it had to literally give Windows XP away for free and extend it's shelf life because Vista wouldn't run on a netbook.

If you have a WiFi router, DVR, DVD player, smart TV, satellite set top box, cable set top box or any number of other devices in your home then the chances are it's running either a Linux or BSD based OS.

Android/Linux smart phone deployments dwarf all others in the smart phone market. Even Apple's iPhone is dwarfed by Android when all Android distributors are counted as one. And Windows Phone 7? Hardly a blip on the radar.

It's true all manner of malware exists for Linux. However Linux has a different approach to dealing with malware. Yes the people sticking their heads in the sand are not helpful. Linux does have anti-malware software built right into the kernel. It's call Apparmor. It works a bit more like a white list of software rather than the black list Windows anti-virus vendors try to use. This is used in combination with community vigilance. One of the great advantages of Free and Open Source Software is the community gets to see the source code. It can be inspected so the community can determine what it does. So additional anti-virus software on Linux is generally speaking not needed.

With the black list approach you'll always be step behind. And that's just not good enough.

So far as the average office worker or home user is concerned GNU/Linux has all the bases covered when it comes to application software. There are more web browsers, e-mail clients and office productivity sweets than you can swing a cat at. And many of these applications are also available for Windows as well. So migration needn't be a harsh experience.

It's true more specialist bespoke software is not available off the shelf. However that's true of all OS platforms. The clue is in the "bespoke software" part. The Microsoft way is to tell people to use their OS and applications stack no matter what. And indeed Microsoft channel partners follow this mantra. Religiously sometimes. However it is very bad practice to shoehorn every business into the same mould. This is in fact the primary reason why malware is such a massive problem for Windows.

When building bespoke systems it is better to assess the clients actual needs and serve those needs first. When that approach is adopted. FOSS tends to win. Hence the reason why many of the worlds stock exchanges have switched to GNU/Linux. Hence the reason why US drones now run on Linux instead of Windows which could not be properly secured against malware. Hence the reason why many EU government agencies are switching to GNU/Linux and why GNU/Linux is so popular in South America. Hence the reason why something on the order of the top 40 of the worlds most powerful super computers are running GNU/Linux and why most of the top 500 are running GNU/Linux.

And then there is the tablet market. Which very closely resembles the smart phone market. ARM based devices running Android/Linux.

At this stage in the game anybody claiming Linux is not a viable contender is just delusional. The last market Linux has left to conquer is the desktop. Which is ironically becoming less relevant every year if the pundits are right. The only market where Windows has a strangle hold is the desktop/laptop market. Linux is slowly gaining ground. Microsoft is slowly losing.

Pageviews by Browser 7 Jan 2010 - 5 Feb 2012

Interested In Linux? Here are a few resources.

• The Linux Foundation
• The GNU Operating System
• The Gnome Foundation
• Debian
• Ubuntu
• Fedora
• Nixie Pixel

Incidentally if you're using Google or Facebook. You're a Linux user already. Something to think about people.

CarrierIQ - FOSS Wouldn't Have Stopped It!

There seems to be quite a bit of fuss going on around this whole CarrierIQ business. Specifically there seems to be some sort of misconception that a fully open source operating system would have some how prevented CarrierIQ from being used. What complete and utter nonsense. Lets consider how this software was discovered.

This thing was detected by security researchers. How many smart phone users out there are security researchers? How many smartphone users out there in the real world actually care what diagnostic software is installed on their "phone"? And now that we know about CarrierIQ, how many Android or Apple smartphone owners are going to do anything to remove it? Almost none is the answer to all questions. Just a teeny tiny minority of people using these devices understands the inner workings enough to even think to look for this sort of activity.

In deed one of the security researchers who discovered CarrierIQ, only found it because he was tracking down the source of some data packets moving across his companies networks that shouldn't have been there. The implication being if he hadn't noticed the rouge data packets. He wouldn't have found CarrierIQ.

Now lets consider how a fully open source OS would have helped. Could Google have reasonably stopped HTC, Samsung or Motorola from installing CarrierIQ? It's doubtful. If Google aren't involved in the installation of this rootkit then I see no way they could have stopped it. Even if it had been installed as a standard default app. Most people still wouldn't have noticed it. And even if they did. They likely wouldn't have done anything about it. The description would have read something like, "Reports performance metrics back to manufacturer for support purposes". Most folks would like then have considered it a necessary technical component and left it well alone. I mean I let my Ubuntu desktop report back to Canonical.

So in the end CarrierIQ would still be there. Most people would do nothing about it. As they are doing now.

If Google had created their own performance monitoring software could they have stopped this? Well no. Android is open source or at least mostly open source. And as with most general purpose operating systems today, Android is modular. That means any component can be changed out for an alternative part by those who have the know-how and will to do so. So Samsung and HTC could still be spying on you.

There is also another issue to look at. Data security on a network. How does open source software protect your data against monitoring once it leaves your phone? The phone companies know who their customers are and who's calling who, who's texting what etc. And it's not just the phone companies. Go talk about any subject or product on Facebook and then watch as the adverts you're served up on web pages match exactly what you were discussing ten minutes ago.

So what exactly is the fuss about? Are people still under the illusion they have some sort of privacy left in this world? Privacy died when the art of "data-mining" was discovered.

....

Just one more thing before the lights go out. I noticed a few folks crowing about how Windows Phone 7 devices don't have this rootkit. Well no they don't it seems however who needs a rootkit to ruin your day when Microsoft are involved. Sidkick, Office 365/BPOS, Windows, Xbox malware. Enough said.

Users Don't Know What They Want

I was reading this article which quoted a comment by Richard Hughes. "User don't know what they want". Well excuse me for being to retarded to write an advanced GUI desktop on my own. But I can choose what to eat for breakfast in the morning. I manage to dress myself. Go out to work. Make it through the day and get home safely. All on my own.

When I bought my current PC. A Dell Dimension XPS 700. I did that on my own as well. I even paid for it. With a credit card. When I decided to make the move to GNU/Linux full time. I decided on my own to do it in stages. The first thing was selecting a GNU/Linux distribution. I experimented with Fedora and openSuSE. Then discovered Ubuntu. Now since I was using the hybrid hardware/software RAID array built into the 700 series system board the installation of a Linux based OS back then wasn't straightforward. Dmraid wasn't installed or configured by default on any of the distros I tried. And that's actually part of what drove my decision to go with Ubuntu.


Being stupid, I couldn't get dmraid working in either Fedora or openSuSE. Both distros use RPMs. Back then RPM hell was a term many people came to know. Basically there were dependency issues. Ubuntu with it's debs was better organised. No dependency issues. Although configuration was still a problem. But with a little research into the Ubuntu documentation I found what I needed to get it all up and running. Which meant I could now dual boot. Until that point I had been boot Linux from a USB drive.

So I'm not smart enough to write my own version of Gnome. I am smart enough though to know how to problem solve. How to make decisions. I know what works for me and what doesn't. Windows XP with all it's issues and problems wasn't working for me any more. So I made the sensible decision to find an alternative. What do you think I'll do now that Gnome doesn't do the things I want it to do? Maybe I'll find an alternative. It would seem to be the sensible thing to do.

When ever a software developer, a programmer makes a comment like "users don't know what they want". It's a clear sign something is going very wrong in that software project. Which reminds me of another comment I read once in a article. Linus Torvalds once said something like "he who writes the code gets to decide". Meaning ultimately programmers participating in free open source software projects are the ones who decide which features to include and which features to drop. Which is fine. Until that is the software you are writing targets a user group beyond other programmers. And that's what desktop environments do.

Kernel developers like Linus Torvalds have a certain luxury of rarely having to interact with desktop users. The concerns of the desktop user are rarely the concerns of the kernel developers. In a sense what kernel developers do is invisible to desktop users. So we desktop users don't complain very often when they drop or replace features. Desktop developers however don't have that luxury. As Canonical/Ubuntu and KDE found out. Gnome should be learning lessons from these two groups.

When KDE 4.x series was released there was uproar. The majority of users hated it. Some of there anger was squarely targeted at being hit with the unfamiliar. The very same issue we have to overcome to get people to use a GNU/Linux based OS in the first place. Anybody would think we'd know better. Right? A lot of the anger however was squarely down to the fact KDE had changed things too much. Now so far as I know, KDE developers aren't known for being polite about people who criticise their work. But after all the bitching was done they knuckled down and started fixing the things people were complaining about. As a result KDE is now a more pleasant desktop environment to use with some pretty cool features. Everybody's a winner!

Canonical has experienced similar anger spat in their direction for daring to force Unity on it's user base. Unity started life as the interface to Ubuntu Netbook Remix. And it's not hard to see why Canonical would think Unity would work well on a netbook. Small displays means you have to be economical with the display. Low powered CPUs meant not much was being done by way of multitasking. But on a desktop? These just aren't considerations that are any where near the top of the list of all the things to be considered. However Canonical would not be deterred. It rolled out Unity.

Most people say Unity was rolled out too soon. It wasn't finished. And indeed they say the same of Gnome Shell. Rather than bitching or insulting it's user base though, Canonical it seems would rather just make Unity better. 11.04 delivered a stale turd of a GUI. With 11.10 Unity was now running atop the new and improved Gnome 3.x. They fixed some of the annoyances. Made the dash useful. 12.04 will focus more on stability and polish. Basically KDE and Canonical listened to their user base. And because they listened they could fix the problems that were pissing people off.

Listening and understanding users is the most important thing a developer working in the user space can do. If your not writing software people want to use then your playing to an empty house. I'd hate to see the Gnome Foundation playing to an empty house. Gnome has been good to me over the years. It's been relatively hassle free. Simple and easy to use and configure. It's developers need to respect the users and listen to what they are telling them.

It would be a shame to see Gnome implode and be crushed by the weight of it's own foot print. There are plenty of alternatives. LXDE, XFCE, KDE and Unity to name but a few.

The Nature of Metro?

A few posts back I theorised Metro was either just a new skin for IE and a tieling mode for Windows window manager. Well here's the proof.

http://www.techrepublic.com/blog/window-on-windows/tweak-windows-8-to-remove-the-metro-interface/5077?tag=nl.e101

The registry hack to switch off Metro and restore the Windows 7 task bar with start menu has already been found. Microsoft should just be distributing this as a service pack or add-on of some kind. Not a whole new OS.

Ubuntu Natty and Unity - Update

The stability gained from removing MediaTomb and disabling the screen saver is short lived. So Boinc is going to have to go. Although I suspect Unity is the real culprit. I've seen some reports that 11.10 Beta 2 is quite good. I might just go a head and install that. Or I could ditch Unity completely. Set up the Gnome 3 PPA and install Gnome Shell/Gnome 3. Alternatively I could go with KDE or LXDE.

I could of course simply ditch Ubuntu and opt for Fedora or Debian. I mean it's not like I'm a total newbie dependant on Canonical doing everything for me. I've had bad experiences with distros in the past that used RPMs. So Debian might be more up my street. It is after all what Ubuntu is based on.

Microsoft Sponsored FUD?

Well here's an unexpected blast from the past. Which looks like Microsoft sponsored FUD. Windows Phone 7 suddenly gets some press coverage when everybody else had written it off and considered it a dead end. Just like Sidekick and Kin. So why is cnet suddenly covering Windows Phone 7 now?

Well unless you've been living in a cave you can't have failed to notice the Windows 8 hype machine starting to roll. The thing about this article that sparks suspicion other than the coverage of a virtually dead and buried mobile platform is that it quotes a study. A study which claimed the majority of the smart phone owning public are considering an Android based device. But yet the author chooses to focus on the Windows Phone 7 numbers? Why? Reads like FUD and propaganda to me.

Windows 8 and UEFI

I generally consider Windows 8 to be vapourware. There's no actual proper working copy of it yet. Although we do now know there is at least a developer version. Which Microsoft showed off recently. Which in turn led to much fan fair and excitement in some corners around the new Metro interface and it's Metro Apps. And I admit Metro does look good. So where's the catch?

Well firstly from what I can figure out, Metro is actually a new look IE designed to work with HTML5 apps wrapped up in an application wrapper. Which is actually how many Android and iOS apps are built. So nothing new there. Metro is basically a gimick. And gimicks are used to distract peoples attention from the small print. So what's in the small print of Windows 8?

UEFI is! UEFI is the proposed successor to the ageing BIOS. Microsoft is requireing that all OEMs and system builders participating in the Windows 8 logo program have a particular feature of UEFI enabled. This feature basically locks down the system so that it will only run approved OSs that have been signed with special security keys. Which isn't a problem if you're happy to just accept whatever Microsoft offer you.

It is however a problem if you like to tinker with your hardware. Remember you pay for the hardware. You own it. Hardware is not licensed unlike software. So surely it should be up to you what OS you choose to run? Well if Microsoft has it's way and OEMs ship their PCs with this new UEFI feature enabled, tinkers will no longer be able to use main stream OEM hardware.

UEFI with it's cripple ware feature enabled will require any OS you choose to install to be signed with those special "security" keys. Which poses problems for OSs like GNU/Linux and FreeBSD. It'll also create problems for project like the Haiku OS. A free open source version of the now dead BeOS. Developers of these operating systems would need to get every OEM to sign their OSs.

Which means at the end of the day UEFI basically excludes homebrew OS from being developed and run on mainstream hardware. Some people think this might violate EU competition rules.  I don't know how true that is. What I do know however is that this makes me think Microsoft are getting really desperate and resorting to some of Apple's dirty tricks to protect their monopoly. Apple use a similar feature of EFI to lock OS X down to Apple hardware.

http://www.uefi.org/home

http://www.theregister.co.uk/2011/09/21/secure_boot_firmware_linux_exclusion_fears/

Ubuntu Natty and Unity - Update

Just a quick up date. So no screensaver enabled, no MediaTomb and guess what? No crashing.The PCs been running all night quietly crunching numbers with Boinc. And it's still usable the next day. The system monitor does show one "zombie" process though, zeitgeits-datah. I'll need to do more research on that. But whatever it is. It's not stopping things from running smoothly.