Ubuntu and NVIDIA - header problem

I recently installed a Ubuntu software update, which made use of a new Linux kernel.

On reboot, I found the GUI screen to be in this huge, clunky graphics mode, where only about 1/3 of my screen was now visible.

It was obvious something was different about usage of my NVIDIA graphics card.

I did a lot of research, and finally got it working again - until the next update that included a new kernel.  Same goes for upgrades - they also broke my NVIDIA graphics card normal operation.

One of the things my research led me to was the Bug Reports for Ubuntu.  The developers have made a deliberate decision, after identifying the problem with the updates/upgrades to kernels which causes the NVIDIA driver to not function correctly, to not include some header files that are needed.  It appears they are being 'purists' instead of making a simple addition to the distribution that would prevent this problem.

As it turns out, NVIDIA drivers need to be built for the specific kernel that is being run.  And, in order to be rebuilt, NVIDIA needs the new header files (have a look at /usr/src/linux-headers* and you should see some linux-header folders) - it needs the source header files for the new kernel, and these new source header files are deliberately not being included in the Ubuntu update/upgrades, even though the developers know this is a problem - they are willing for you, and thousands of other users, to not have a working computer and expect you to fix this problem yourself.

So - this is how I went about fixing my problem.
Note: here's a useful link.

I didn't know where my original NVIDIA driver was at, and didn't really care to find it - I wanted to make sure I was running the newest drivers.

From a high level, this is what I did - I am not saying this will work for you, and in fact it may cause you problems with your system.  More detailed information will follow.

Outline:
( I am not an expert.  I am not responsible for any problems caused from use of this information.  You assume all risk.  If something goes wrong, I will not help you.):

• I downloaded the NVIDIA driver for my card and Operating System.
• I changed the NVIDIA driver script to executable
• I went to a user console (this is not a X-Terminal) - NVIDIA wants the X GUI stopped.
• stop the X GUI
• I moved to my download directory where I placed the NVIDIA driver
• I ran the NVIDIA driver script - it will report an error for missing source linux-header files
• I noted the missing source linux-header file
• I downloaded and installed the missing linux-header file
• I ran again the NVIDIA driver script - this time it succeeded
• I rebooted

From a detailed level, this is what I did - I am not saying this will work for you, and in fact it may cause you problems with your system.  More detailed information will follow.

Details:

 ( I am not an expert.  I am not responsible for any problems from use of this information.  You assume all risk.  If something goes wrong, I will not help you.):

1. I made a backup of my Operating System before proceeding, in case something went wrong), and placed this backup onto a different hard drive. I know how to reinstall this backup successfully, having tested doing so in the past. 
2. First, I downloaded the NVIDIA driver for my operating system (64 bit), and for my graphics card (series 500), from this site (current as of 12/3/13).  
1. My default download location is ~/Downloads.  http://www.nvidia.com/Download/index.aspx
3. I went to a user console (this is NOT an X-Terminal)
1. I Depressed simultaneously these keys: Ctrl-Alt-F1
2. I logged into my computer with my userid and password
4. I stopped the X GUI - (NVIDIA shell script requires this)
1. sudo stop lightdm
5. I changed to my Downloads directory where I had placed my NVIDIA driver
1. cd ~/Downloads
6. I changed the NVIDIA driver script to be executable (my downloaded driver was named NVIDIA*.run   - the * is mine, to keep from typing the whole string)
1. sudo chmod +x NVIDIA*.run
7. I executed the NVIDIA*.run shell script
1. sudo sh NVIDIA*.run
2. I selected Yes to all questions.
3. This should produce an error message - if it had not,  I would have tried rebooting now, as I was probably good to go)
8. I Noted the error message where it mentions the 'linux-headers' source file - this is the header files the NVIDIA driver needed to be download and installed
9. I obtained the specific linux-header source file identified by the NVIDIA.run script and installed it
1. sudo apt-get install linux-headers-THE-FULL-NAME-I-OBTAINED-FROM-ERROR-MSG
2. Example: sudo apt-get install linux-headers-3.11.0-12-generic
10. I re-ran the NVIDIA*.run shell script
1. sudo sh NVIDIA*.run
2. I selected Yes to all questions
3. It reported success
11. I rebooted my compuer
1. sudo reboot
12. My computer rebooted, and my problem was fixed.

Upgrading my system: Ubuntu 11.10 to 12.04

Upgrading Ubuntu from 11.10 to 12.04 and migrating from SSD to larger HD

June 5, 2013


I have multiple computers spread throughout the house  Arrangement:

• 3rd floor - I day-trade the ES e-mini S&P 500 futures.  So I have a lot of redundancy in my office.  Currently:
• Laptop running Windows 7
• Laptop running Windows Vista
• Windows Vista desktop
• Ubuntu desktop
• 2nd floor - A living room Ubuntu laptop which shows weather radar from www.weather.gov for our Atlanta GA area.  Also has XBMC and a big analog clock which can be viewed from anywhere.  Also an Ipad and a Chromebook on this floor.  An old laptop on the porch for streaming music, which is running Windows XP.
• 1st floor - A "Media PC" from HP on the first floor.  This is also connected to our antenna for local over-the-air tv reception via a USB to Antenna device, and an internet connection for streaming.  This drives a large HD tv, which is our primary entertainment system.

I wanted to make some changes to my Ubuntu (16GB memory 1066 Mhz, i5-2500 cpu @3.30GHz x4, Asus P8Z68-Pro):

• Upgrade the Ubuntu OS from 11.10 (which is no longer supported) to 12.04 (which has LTS - long term support).
• Upgrade from a small SSD (120GB)to a large HD (3TB).
• Swap out a network switch.
• Add a VirtualBox, and install Windows 7 on it.

Upgrading OS

• The first thing I wanted to do was to upgrade the Ubuntu OS from 11.10 to 12.04.  I downloaded the 64bit versions of Ubuntu 12.04 and 13.04.  I created bootable DVDs (AKA LiveCD) for each of these images.
• Powered down the system.
• I disconnected all of my extra hard drives (5), leaving me with just the one OS hard drive connected.
• I connected up the new HD that was empty.
• I booted from the 13.04 LiveCD to use the 'dd' utility.  I did this from 13.04 in case there were any improvements made to 'dd'.  In case you don't know, 'dd' makes a complete image of a hard drive, which is what I wanted to do.
• Powered down the system.
• I disconnected my old SSD, removed the LiveCD from the CD drawer.
• Powered up to the new HD.
• Logged in, had a look around, took a break.  Later returned and looked around a little more, ran some intensive apps (trading app), saw everything was good.
• Powered down
• Reconnected the extra HDs (5), and rebooted, again poked around - everything looked good.
• Inserted the LiveCD for Ubuntu 12.04 LTS.
• Booted to LiveCD, found problems with 'channel 1' error. Searching the Internet, saw that I needed to change a boot option at the 'Ubuntu' splash screen, so selected the option for 'nomodeset', rebooted and was good to go.
• Upgraded to 12.04
• Lost all my apps, so had to reinstall them.  I already have a system backup on a different HD, and I still have my original SSD, so no concerns about losing anything.
• Next I wanted to make use of the extra disk space, so I installed 'gparted' on to Ubuntu.
• I increased the OS root partition to 1TB.
• Rebooted and checked around - everything functioning correctly.
• I decided I wanted to make a new partition on this hard drive, so I created a new 1TB partition, which would be mounted at /home.
• created a new partition using gparted, making it 1TB.
• created a new mount point at /newPartition
• made an entry in /etc/fstab to mount this new partition
• used 'rsync' to make a copy of /home to /newPartition
• poked around /newPartition, saw that everything was good.
• renamed /home to /oldHome
• renamed /newPartition to /home
• modified the /etc/fstab to change the mount point for /newPartition to /home.
• rebooted, confirmed the new /home partition was functioning correctly.
• Left, for now, the /oldHome in place, but plan on deleting it in the future - it's still mounted and accessible.
• powered down, connected up all the extra HDs (5).
• booted up, verified everything was good!

VirtualBox

• Decided I wanted to add VirtualBox to Ubuntu system, and install Windows 7 onto it, as sometimes I need access to Windows, and would like to be able to get to it on Ubuntu as over the weekend when I'm not trading, I take my other Windows system down and don't want to power it back up.  Also, want to be able to try out some new Windows trading software, so installed VirtualBox.  Had a few issues with screen size, did some Internet research, found that I needed to install the 'Guest Additions', so did that, and my screen problems went away.
• Surprised and pleased at how well this VirtualBox and Windows 7 installation is running - can't tell any performance hits on the Ubuntu system.

A Home Security System - BeagleBone Black getting started

BeagleBone Black - Getting Started

Keywords: BeagleBone Black, Home Automation, Home Security, Python, Raspberry Pi, Tkinter, 

Updated June 5, 2013 (Table of Contents)
May 18, 2013
Next
Table of Contents

Note that I'm on an Ubuntu system, not a Windows system.

I received my BeagleBone Black (B3) yesterday.  Now I'm trying to figure out how to use it!

Per the included card that came in the B3 box, I noted, under Tethered to a PC:

• Connect the USB mini cable to BeagleBone Black.  OK, did that.  They supplied the cable - nice!
• Connect the other end of the USB cable to the PC.  OK, did that too.  
• Look for a new mass storage drive to appear on the PC.  OK, it's there.
• Open the drive and click on START.htm.  OK, there it is.  Clicking...
• It opened a web browser page, in URL:  file:///media/BEAGLEBONE/START.htm , and presented the "Getting Started" page.  Nice!  I skipped the step 2, "Install drivers", as it noted: "Driver installation isn't required, but you might find a few udev rules helpful".  This has no meaning for me, as I don't know what udev is or the rules they are talking about, so I'm going to skip for now, since they said it wasn't required.  I can always do later...
• Step 3, Browse to web server on Beagle.  I did this, and observed the web page from my B3's server on http://192.167.7.2.  

• On this page, under Troubleshooting, it says not to use Windows Explorer.  That's OK, I'm not using it on my Ubuntu node that I'm working from, I'm using Chrome.  I'm not Troubleshooting either, as I'm not having any problems.
• The next section is "Update board with latest software".  Here, I got confused.  I'm on Ubuntu, but everything here appears to be oriented towards assuming I'm on a Windows node.
• It says" Download "Angstrom Distribution" from http://beagleboard.org/latest-images". 
• Notes - 'due to sizing necessities, this download may take 30 minutes or more".  OK, it takes whatever it takes, no issues here.
• The file you download will have an .img.xz [sic]extention.  This is a compressed sector-by-sector image of the SD card.  OK, no issue here for me.
• Then it says, Step#2, Install compression utility, download and install 7-zip.  OK , then the screen clearly shows a Windows "open file security warning, publisher could not be verified, run?"  Well, I'm not running on Windows, so now what do I do?  I read along further...
• Step #3, Decompress the image.  Use 7-zip to decompress the SD card .img file.  OK, still showing a Windows screen here.  Can't do that....so reading further to see if I can figure this out, or see what else may present itself...
• Step #4, Install SD card programming utility.  Download and install Image Writer for Windows, be sure to download the binary distribution.  OK, can't do that either, so let's continue down to see what might present itself and clear this up....
• Then an odd note, after all this Windows stuff:  Some general help on programming SD cards can be found on the Ubuntu Image Writer page.  OK, that sounds like me, so let's have a look; but what about all the zip unpacking, etc?  I'll have to let that be unresolved for the moment.  In the meantime, I'm over at the Ubuntu site, and note this "describes how to install from a .IMG file', and  about "downloading IMG image to a device that your computer can boot from", so maybe I'm onto something here.  So, for now, I'm assuming this is what I need to do.  
• Returning, for now, to the B3's Update Software section, I note "Connect SD card to your computer", and I'm wondering, which computer are they talking about?  My Ubuntu computer, or the B3?  Reading further, it says "Use the provided microSD card to SD adapter or a USB adapter to connect the SD card to your computer.  Well, there wasn't any provided "microSD card to SD adapter"!!!  The uSD slot (under board) is empty, and other than the USB to micro USB cable provided, there's nothing else provided.  So, something wrong here!  Reading further....
• Step #6: Write the image to your SD Card.  OK, that clears up one question, where I didn't know which computer they were referring to - my Ubuntu computer, or the B3.  It's now clear they are referring to my Ubuntu computer.  OK, got it - I'm going to program an SD card, using the Ubuntu instructions noted above from the Ubuntu web site, with the .IMG file.  OK, understood.
• Step #7: Eject the SD card.  OK, got it.
• Step #8: Boot your board off of the SD card.  They then give detailed instructions on what to hold down while booting from the SD card.  One thing that concerns me though is that it doesn't say how big an SD card (in capacity) I need.  It also says "this can take up to 45 minutes."!!!
• Well, looks like I'm stopping here until I can go to the store to buy a 4GB SD card.  While I'm doing that, I'll go ahead and download the BeagleBone Black latest-images (Sngstrom Distribution (BeagleBone Black) 2013-05-08.  Well, I first clicked on the 'torrent', but it came up with a warning that torrent would also mean I would agree to upload.  I don't want to do that off of this computer, so I canceled that, and used the normal download instead.  I'm also going to get a 5VDC power supply.
• Hmmm, can't seem to find the specs on the power supply physical dimensions needed.  After much searching, I found it embedded in the document:  The connector used is a 2.1MM center positive x 5.5mm outer bar
• I want to power down the B3 - I never leave new electronic devices powered up and unattended, so I want to power it down - how to do that?
• OK, here it is:  A power button is provided near the reset button close to the Ethernet connector....f you hold the button down longer than 8 seconds, the board will power off if you release the button when the power LED turns off. If you continue to hold it, the board will power back up completing a power cycle
• Yep, that did it.

UBUNTU

• So, this is what I did to install an updated image via Ubuntu.
• First, I downloaded the image on to my Chrome browser into the Download directory.
• Then I double clicked the .xz file.
• This brought up an Archive Manager window, where I selected the 'Extract' button.  In this case, I created a new folder and labeled as pertaining to BeagleBone Black and the date, then I extracted to this folder.  This created a .IMG file.
• This is the file that needs to be transferred onto the SD card.
• Per the Ubuntu post here, this page describes how to install from a .IMG file.  
• It says to 'write the downloaded IMG image to a device (my SD card in this case) to a device that [my] computer [my B3] can boot from.  It also notes that this action will destroy any data on the destination SD.
• There's a lot of good information on this page, so I would suggest that you view it and understand it before proceeding.
• The steps given:
• 1. Download the .img file.
• 2. Install the usb-imagewriter package
• 2.1 If your release does not include this, download it from Oliver's PPA
• 2.2 if the imagewriter fails to launch, you may need to install python glade2 support.  Install the python-glad2 packge or Run sudo apt-get install python-glade2
• 2.3 If your release does not include it and you are runing Ubuntu 9.04 Jaunty Jackalope then run this command from the console: sudo apt-get install usb-imagewriter
• 3. Oopen Applications->Accessories->Image Writer
• 3.1 from teh command line, from the console:
• sudo imagewriter
• On some usb-imagewriter versions (console command:imagewriter) the applications fails to write if the image path contains blank spaces, exiting with "IndexError: list index out of range".
• 4. Insert your flash media
• 5. Sleect the downloaded file and flash device, and click "Write to Device"
• 6. REmove your device when the operation is complete.
• Well, in my case, I went to 'console' and typed in 'sudo imagewriter', and it launched, so I didn't have to do any of the installation stuff before.  I probably installed this previously.
• When I placed the SD card into my Ubuntu's SD slot, I was expecting to see it in the File Manger, but I never saw it.  I looked in the folders Mnt and Media, but didn't see it there either.  So, on a whim, I launched 'imagewrite' and sure enough, it found it.  So I'm assuming this didn't have any type of file system created on the SD, so nothing was seeing it.
• Running 'imagewriter'
• it presented an interactive window, where it prompted me for the Write Image to select.    I used this to select the file from the folder I created previously (above).
• From the 'to' listbox, I selected the SD card I had inserted.
• I selected 'Write to Device'
• I OK'd the warning that says it will destroy all data on the device (which as it turns out, is mounted at /dev/sde).
• And, at 7:21 PM, it started writing.  Knowing that Beagle's site said it may take 45 minutes, I'm expecting a loooonnnnnggggg wait.  So far, I don't see anything, so I'm going to check the process running on my system to see if I can see it writing and consuming CPU cycles.
• Update 6/5/13  Sorry for the long delay.  Had some other things pop up, and just now getting back to this.  So, to resume from the post above:
• I had some problems getting the write to complete without error.  I suspected it was the SD card, so I removed it and tried another one, and the replacement worked fine.  However, it was too small overall, so I wouldn't be able to use it for the final installation.  
• My system, which I built from components (my first computer, in 1978, was built from discrete components, such as transistors, capacitors, etc, except for the mother-board.  This was a Heathkit, H11, which incorporated a PDP-11/03 motherboard from DEC.  It cost me about $3,500!  In 1978, you could buy a CAR for this amount of money!  But now, I just buy the components and plug them in - no more putting the springs under a keyboard, no more solenoids for a paper tape punch, no more flyback transformer for the display monitor!).  I did not include a separate SD card reader, instead opted to use a USB SD card reader.  On a whim, I decided to try a different USB SD card reader I had with the problem SD card.....and the SD card worked fine!  So, it was a combination of the USB SD card reader device and the SD card itself that was causing the problem.  Switching to a different USB SD card reader and the same SD card worked great.
• So this part of the project is complete!

A Home Security System - Resources

Resources

Update: May 18, 2013 (Table of Contents)

• BeagleBone Black
• Python

A Home Security System - Construction Blog

Construction Blog

Keywords: BeagleBone Black, Python, Tkinter, Raspberry Pi, RPi, Arduino, Security System, Home Automation

Update: June 5, 2013. (Table of Contents)

• Updated this entry (at the bottom) with SD card info.  http://bbqandbanjos.blogspot.com/2013/05/a-home-security-system-beaglebone-black.html
• My Host system changes:
• Installed 'Git' onto my system for Source Code Management.
• Had been running on a small Solid State Hard drive, but have decided to move to a larger physical hard drive.
• Decided to migrate from Ubuntu 11.10 to 12.04 which has long term support.
• I used Linux's 'dd' command to image the new drive.  See details here.
• Noted that the Python version installed on the BeagleBone Black (B3) is not Python3, which is what I was planning on using.  Will have to investigate whether I can upgrade Python on the B3 to Python3, or will have to move back down to Python 6 or 7 - not sure which is on B3 at the moment.
• Will be out all next week, so I don't expect to make any additional postings.
• 
  

Update: May 18, 2013 (Table of Contents)

I have been pretty busy on the project.  I've been working on the following elements:

• Learning about the BeagleBone Black (B3) platform.  I decided to move the project off of the Raspberry Pi (RPi) and onto the B3.  I'm concentrating right now on the following aspects:
• How to interface with the onboard mux.  This is used to specify which channels, or pins, are being addressed.  
• Learning Python.  I've really been enjoying this!  I've been programming for a while: Fortran (1977), Assembler (1978), C (1983), C++(1995), Java (1999), Arduino (2004?), and Python3 (2013), so the move to Python3 hasn't been huge, but it's been different never-the-less.
• Learning Tkinter.  This is one of the main standard GUI interfaces into Python.  
• Designed a loose GUI concept mockup
• B3.  Not yet deep enough into this, but I'm planning on programming it using Python3.  
• Just received it on 5/17/13.
• Concentrating on getting it up and online.
• Concentrating on Mux addressing via Python3.
• So today, after finishing on this project update, I'm going to start with the B3 and getting it online, then accessing it's pins via short Python3 programs.

A Home Security System - Sensors OOD

Sensors OOD

Keywords: BeagleBone Black, Python, Tkinter, Raspberry Pi, RPi, Arduino, Security System, Home Automation

March 25, 2013 (Table of Contents)


This system will be Object Oriented (OO).  This page will be concerned with the Object Oriented Design (OOD) of the Sensors.  It will make use of Polymorphism.

The root type: Sensor

A Sensor Object:

• It has the following Attributes:
• Name
• Zone
• Description
• It has the following States: 
• Ignored
• Alarm
• Alert
• Normal
• It has the following Transition Listeners
• ListenerOnTransitionToAlert
• ListenerOnTrasitionToAlarm
• LietenerOnTrasitionToNormal
• ListenerOnTrasitionToIgnored
• It has the following Listener Methods
• RegisterAlertListener
• NotifyAlertListener
• RegisterAlarmListener
• NotifyAlarmListener
• RegisterNormalListener
• NotifyNormalListener
• RegisterIgnoreListener
• NotifyIgnoreListener
• It has the following Sensor Control Methods
• SetSensorToAlarmMode
• SetSensorToAlertMode
• SetSensorToIgnoreMode
• It has the following Sensor Test Methods
• TestSetSensorToAlarm
• TestSetSensorToAlert
• TestSetSensorToNormal
• TestSetSensorToIgnore

Sensor Children Types:

• A Magnetic Contact is a type of Sensor.
• It has the following states:
• Contact State
• Open
• Closed
• It has the following Sensor Normal Methods
• DefineNormalAsOpenContact
• DefineNormalAsClosedContact
• It has the following children types:
• A Door Sensor is a type of Magnetic Contact Sensor
• A Window Sensor is a type of Magnetic Contact Sensor
• A Voltage Transition Sensor is a type of Sensor
• It has the following state transitions:
• Transition from Voltage Low to Voltage High
• Transition from Voltage High to Voltage Low
• It has the following Sensor Normal Methods
• DefineNormalAsLow
• DefineNormalAsHigh
• It has the following children types:
• A Doorbell Sensor is a type of Voltage Transition Sensor
• A Carbon Monoxide Detector Sensor is a type of Voltage Transition Sensor 
• A Smoke Detector Sensor is a type of Voltage Transition Sensor
• A Water Detector Sensor is a type of Voltage Transition Sensor
• A HVAC Temperature Controller Sensor is a type of Voltage Transition Sensor
• A Household AC Power Sensor is a type of Voltage Transition Sensor
• To be classified:
• Networked Sensor
• Video Camera Sensor
• Access Keypad
• Door Speaker/Microphone

A Home Security System - Web Pages

Web Pages

Keywords: BeagleBone Black, Python, Tkinter, Raspberry Pi, RPi, Arduino, Security System, Home Automation

May 18, 2013 (Table of Contents)

• Reviewing whether I will use a client/server GUI or web pages.  Not yet decided.

March 25, 2013  (Table of Contents)

Web Page

Users

• Add User Web Page
• Delete User Web Page
• Modify User Web Page

Sensors

• Add Sensor Web Page
• Delete Sensor Web Page
• Ignore Sensor Web Page
• Modify Sensor Web Page

Control Elements

• Add Control Element Web Page
• Delete Control Element Web Page
• Ignore Control Element Web Page
• Modify Control Element Web Page

Monitoring

• System State Web Page
• Sensor State Web Page
• Alarm State Web Page
• Log Web Page

Messaging

• Add Message Recipient Web Page
• Delete Message Recipient Web Page
• Modify Message Recipient Web Page

Video

• Video Camera Web Page

A Home Security System - Control Elements

Control Elements

Keywords: BeagleBone Black, Python, Tkinter, Raspberry Pi, RPi, Arduino, Security System, Home Automation

March 25, 2013   (Table of Contents)

• Control Element Types
• X10
• 120 VAC Control
• Door Lock
• Relay Contact Control
• Door Speaker/Microphone (e.g., communicate with someone who rang front door).
• Alarm Siren
• Alert Speaker
• Video Camera
• Networked Control
• HVAC On/Off
• HVAC Damper Control (Open/Close)

A Home Security System - Sensor Types

Sensor Types

Keywords: BeagleBone Black, Python, Tkinter, Raspberry Pi, RPi, Arduino, Security System, Home Automation

March 25, 2013   (Table of Contents)

Sensor Types

• Window 
• Door 
• Doorbell
• Networked Sensor
• Video Camera
• AC Household Power
• Household HVAC status
• Water Leak
• Temperature 
• Smoke
• Carbon Monoxide
• Access Keypad
• Door Speaker/Microphone

A Home Security System - Users

Users

Keywords: BeagleBone Black, Python, Tkinter, Raspberry Pi, RPi, Arduino, Security System, Home Automation

March 25, 2013   (Table of Contents)

The [Name] has three (3) types of users: An Administrator, a Commander, and a Monitor, all of which must login to use and/or view the system.

• An Administrator has all capabilities of a Commander and a Monitor, plus the ability to:
• Configure the system.
• A Commander has the capabilities of a Monitor, plus the capabilities to:
• Set Sensor State
• Acknowledge Alarms and Alerts
• A Monitor has the capabilities
• Observe the system State Web Page

A Home Security System - Glossary

A Home Security System - Glossary

Keywords: BeagleBone Black, Python, Tkinter, Raspberry Pi, RPi, Arduino, Security System, Home Automation

March 25, 2013  (Table of Contents)

Glossary

• User: any one who has the capability to log in and interact or view the system.
• Sensor Element- an input device that detects a state change.  It has an associated Activity Action to perform.  E.g., magnetic window and door alarms, smoke detector, temperature.
• Control Element - an output device under the control of the system, such as an Alarm Siren, a 120 VAC relay, HVAC, etc.
• Zone - the home is divided up into Zones.  E.g., back yard 1st floor.
• Set Sensor State - set the state of a Sensor to Alarm, Alert, Ignore. 
• Alarm - highest level of capabilities.  Alarm Action performed
• Alert - Alert level of capabilities.  Alert Action performed
• Ignore - Ingore the state of this Sensor.

A Home Security System - Design Narrative

Design Narrative

Keywords: BeagleBone Black, Python, Tkinter, Raspberry Pi, RPi, Arduino, Security System, Home Automation

May 18, 2013  (Table of Contents)

Since the last posting (March 25, 2013), which documented the use of the Raspberry Pi  (RPi), I have become aware of a new platform - the "BeagleBone Black" (B3).

The B3 has the some elements, such as faster and more powerful cpu, but primarily because of the increased number of I/O pins, including analog, that the B3 provides over the RPI.  This change will provide, in my opinion, a superior platform, and ultimately a lower overall cost because of the increased number of I/O pins - I was going to have to implement a separate hardware Mux to get all of the I/O I was going to need for the project, and this was probably going to have to be an Arduino, which incurs it's own associated additional costs.

The costs between the two platforms is minor: $35 for the RPi vs. $45 for the B3.

So, moving forward, I will be using the B3.

Design update:

• GUI Interface

• I have done a mock-up of the GUI For the system.  It's not finished, and it may look completely different at completion, but this is what I'm currently thinking about implementing (see picture below).  This isn't functional - it's just a picture of a Python GUI implemented in Tkinter, but shows the current concept.  It shows a big area that serves as a flashing alarm in the Home area (something in alarm in the home), then the Area involved, then the Zone involved, then the sensor involved.  All will flash (alternate white/red background) until acknowledged, then seal-in to show the state.  A new event (return to normal, or another sensor goes into alarm) will cause it to go back to flashing.  Every action will be performed from this GUI, from any computer on the system or remotely when I'm away, via Smartphone or Internet connected device.  From here I will also be able to take a sensor Inactive (bad sensor), Bypass (we want to have the backdoor open and not alarm), as well as the state of the sensor (Normal, Alarm), by selecting the sensor involved and changing it's state (e.g., if going from active to Inactive, or active to Bypass)
• Sensors will include:
• Window open/close magnetic sensors
• Door open/close magnetic sensors
• Motion detectors (PIR)
• Smoke detectors
• Carbon Monoxide detectors
• Natural gas detectors (not sure if these exist or not yet)
• Water detectors (flooding in basement rooms)
• Doorbell
• Loss of Electrical (office, refrigerators, basement utilities room)
• The security system will divide the home into the following:
• Areas
• Basement (this is a live-in basement with one side exposed to the yard outside; our yard slopes, so three sides are below ground level, and one side is at ground level),  with a den room, full kitchen, bedroom, exercise room, living room, bathroom, with two (2) entrances (one from outside, one from inside) and four windows (two in bedroom, two in living room).  This can function as an in-law suite.  This level also has all utilities, such including Internet and security system convergence, as well as a furnace room where I want to monitor for Carbon Monoxide and Fire/Smoke.
• First Floor.  This floor has three doors: one to garage, one to back yard, and one to front yard.  There are 10 windows and a door on the back, one door on the front, and one garage door that will need to be monitored.  One bedroom.
• Second Floor: There are four bedrooms and a utility room, each with windows and doors, and a central hall, all of which will need to be monitored.
• Attic: There's a furnace up here, so I want to be able to check for Carbon Monoxide leaks, as well as water leaks.
• Garage: Carbon Monoxide, Fire/Smoke.  It also has an outside door, and two car entry/exit garage-doors, along with two windows.
• Outside: Front door area, driveway area, basement area, back yard area.
• Zones
• Each area will be broken down into zones.  For example, the Basement area has the following zones:
• Living room
• Bedroom
• Kitchen
• Utilities room
• Furnace room
• Exercise room,
• Den
• Similar breakdowns from Areas to Zones will be done for each Area.

March 25, 2013 (Back to Table of Contents)

Keywords: Raspberry Pi, RPi, Arduino, Security System, Home Automation

Raspberry Pi

The security system is based on a Raspberry Pi (RPi) device.  It may incorporate the use of an Arduino in order to facilitate work with external hardware devices - this is under evaluation.  

The RPi will provide the following services:

• Web App Hosting via Tornado
• Python program execution
• File system for data retention
• TCP/IP network communications
• Communication with Arduino for hardware communications, via RS232

Python will provide the program Model, View, and Controller (MVC) services.  It will also provide:

• Ability for the Administrator to configure the via a remote web monitoring page.
• Ability for the Administrator to change the state of the system, sensors, and control elements via the remote web monitoring page.
• Ability to monitor the status of the security system.
• Ability to push status information to a remote web monitoring page.
• Ability to send SMS text messages
• Ability to control 120 VAC contacts
• Ability (possibility) to control X10 devices
• Ability to control other devices through relay contacts
• Ability to link to, or stream (undecided) video cameras
• Object Persistence via Shelve
• Web Pages via Tornado
• Web Page push updates via HTML5 Sockets

The Security System

The System

The System shall be provide the following services:

• Screens to support configuration of the system.
• Add, modify, delete Users
• Add, modify, delete Contact Phone numbers
• Add, modify, delete Senors
• Add, modify, delete Control Elements
• Screens to show system state:
• System Status
• Network Connection
• File System
• On AC or Battery Backup
• Watchdog
• Sensor State (for each sensor)
• Normal
• Alert - time of event
• Alarm - time of event
• Ignore - time of event
• Acknowledged - time of event
• Messages (All sensors, Control Elements, System Status)
• Messages displayed and logged
• Events (all)
• Time of event
• A log entry for any change in state of any Sensor or Control Element. Will contain such information as:
• Sensor Name
• Zone
• State Change
• User Login
• User Name
• Failed Attempts
• User Logout
• User Name
• Web Browswer IP addresses served
• Page accessed
• User Name
• Invalid Attempts

The Security System will monitor all sensors for changes from normal state.  On detection of a state change for any sensor, the will perform the assigned action.  Each action is defined for each sensor by the Administrator.  The assigned action will continue until either a timeout for this sensor's action, or a command from a User.

A Home Security System - Table of Contents

Home Security System: Table of Contents

Keywords: BeagleBone Black, Python, Tkinter, Raspberry Pi, RPi, Arduino, Security System, Home Automation


This will be the organized central collection of the information associated with the design, development and documentation of the Home Security System.

Section X: Project - Home Security System

• Update

Section X: BeagleBone Black - Getting Started

Section X: Design Narrative

Section X: Construction Blog

Section X:  Updates

Section X: Users

Section X: Sensor Types

Section X: Control Elements

Section X: Web Pages

Section X: Testing

Section X: Documentation

Section X: Releases

Section X: Resources

Project Update 1 - A Home Security System

A Home Security System

Keywords: BeagleBone Black, Python, Tkinter, Raspberry Pi, RPi, Arduino, Security System, Home Automation

Change to architecture
April 6, 2014 (Back to Table of Contents)

I recently came back to this project after taking a break to do other things.

The current architecture (same as before):

• Python is the primary language
• It will run on a BeagleBone Black

The changes to the architecture:

• Is a distributed application
• Using Kivy for the presentation layer (GUI)
• Monitor will be available on any GUI device (e.g., Linux, Windows, Mac, Android)
• Uses Json for IPC and network communications

Search for a Python Oriented OODBMS

May 18, 2013   (Back to Table of Contents)

• Have decided to go with BeagleBone Black instead of Raspberry Pi.

March 25, 2013  (Table of Contents)

Keywords: BeagleBone Black, Python, Tkinter, Raspberry Pi, RPi, Arduino, Security System, Home Automation

After much research regarding Object Persistence (OODBMS) for Python, I've elected to use the built-in capability of Python Shelve.

Frankly, I'm surprised that there aren't more active Python OODBMS applications!

I investigated the following candidates, all of which must be open-source, and the reason, from my opinion, for my rejection:

• Zodb - overkill for this application
• MongoDB - overkill for this application, not a OODBMS
• Buzhug - not very active, no support for Python 3 yet.
• Pypersyst - not active
• CouchDB - document oriented database
• Objectivity - no support for Python 3 yet, commercial product
• Axiom - not a OODBMS; a mapper between SQL database and objects
• Durus - Requires a download of DurusWorks, which appears to be a complete environment.  For my application, this is more of a challenge than I want to undertake.  It may be a great product though!
• Cog - no support for Python 3 yet.  Does not appear to be very active.
• Poodle - no support for Python 3 yet.
• Dobbin 0.3 - Was a potential candidate.  Says it's compatible with Python 3.x. However, no activity more recent than 11 months ago.

Project - A Home Security System

A Home Security System

Keywords: BeagleBone Black, Python, Tkinter, Raspberry Pi, RPi, Arduino, Security System, Home Automation

 (Table of Contents)

Overview

March 24, 2013
Last updated 5/18/13

• Have decided to go with BeagleBone Black instead of Raspberry Pi.

Last updated 3/25/13

Keywords: BeagleBone Black, Python, Tkinter, Raspberry Pi, RPi, Arduino, Security System, Home Automation

I'm going to be replacing my current home security system with one I'm building.  This will ultimately morph into a Home Automation System.

Here are some of the highlights and design decisions:

• Replace the current security system main board with by a Raspberry Pi.
• It may make use of an Arduino to interface with the various contacts - not yet sure on this approach.
• It will interface to the Internet
• It will have a web server interface, where it can display the current status, as well as accept commands (such as "bypass a specific door alarm as I am leaving it open").
• It will make use of IP Tunneling for web pushes to a browser via HTML5 (web sockets).
• It will feature SMS text messages to registered cell phones
• It will replace the existing commercial security main board
• It will utilize the existing window and door alarm contacts
• It will use Python for the development language
• It will make use of Tornado for the web server
• It will be Object Oriented in it's design
• It will utilize an Object Persistence framework rather than a SQL database.  I am currently researching this framework, and no technology has yet been chosen.  Update 3/25/13 - I have chosen Python's Shelve.
• It will display multiple video cameras, and offer pictures and video retention.
• It will interface to fire, carbon-dioxide and water-flood detectors.
• It will be open source
• It will eventually use thermocouples for multi-room temperature monitoring and possible individual-room control.
• It will control multiple HVAC units for multi-floor homes.
• It will be capable of controlling a BBQ smoker's dampers at temperature, with alarms and alerts.

I will document the design, development and deployment of this system.

My Background: I have significant electrical, electronic, and software capabilities, and this project is within my scope of expertise and experience.

I took a Nerd Test

I took a Nerd Test tonight while waiting on some futures contracts to close.

I made a 93%.

That means only 7% of the population is more 'nerdy' than me.

I think that's good.

Thinking that's good is worth two more points!

7% scored higher (more nerdy),
1% scored the same, and
92% scored lower (less nerdy).

What does this mean? Your nerdiness is:

Supreme Nerd. Apply for a professorship at MIT now!!!.

Link to test

If you are thinking about a Cisco/Linksys router

After having multiple problems with a new model EA6500 router I came across this, which is my problem:  http://community.linksys.com/t5/Wireless-Routers/EA6500-Multiple-devices-under-one-device/td-p/572310

There are, to date, 18 PAGES of people with problems.

Note these problems were reported last year, and still no fix from manufacturer!