Archives

All posts by Lemon Grass

I have lost my wooden spoon
I had it a minute ago while I was stirring the soup
Now I wish to stir the soup again
And it is gone
Gone!
How can this be? I am an educated man.
I have not done much harm today because I have not been out
So then, why should this happen to me?
It is not where it should be. Worse,
It is not where it should not be
It is not in the phonebook
It is not under the cushion
It is not down the back of the fridge
It is not in the Arts pages
It is not on my manuscript
It is not in my boot

I am agitated. I can do without this.
Surely this should not happen to a person in their own home?
With no one about to blame
I have prayed
I have made sudden dashes at things and lifted them up quickly
Before it could get away
It wasn’t there
I have gone into rooms I have not been in and looked
Drawers and cupboards the same
It is not here
It has left the flat
It does not exist
It has been transported
Deported dematerialised – whatever it is called –
Teleported yes!
Even now I suppose it is on television
In a programme on the occult

Very well then, I do not need it
I never liked it
I will get my other my second my reserve spoon
I am not without spoons
I have spoons
I do not need it
I do not need to be agitated about it

See, I now have my reserve spoon in my hand
It is friendly, it acts like wood, I like it
I stir the soup
What is this?
Something in the soup impedes my deft stirring motion
What could it be?
I do not wish to know
I do not wish to know that I am foolish
I’m not having soup now
It is a funny colour
I have somewhere a tin of sardines
I had a tin of sardines, I am sure

I am going to eat out
I am going to a restaurant
I have been in all day
I deserve a treat, the company of my fellows
A glance at the Telegraph
What harm could there be in that?
None. Good. I will go.
I put on my coat successfully, and here are my gloves
Now where is my hat?
It is not where it should be.
I am not looking for my hat
I am not going out
It has been a long agitating day
I am going to bed
The bed is in the bedroom
Surely that is the case?
It was there when I looked for the spoon.
Was it there when I looked for the sardines?
Oh now I am being foolish
Boldly, I open the door and go into the bedroom
Oh, there is my hat.

– Alan Jackson

The VirtualBox Guest Additions are software services which allow client operating systems to interact with the Virtualisation environment, breaking the limitation of strictly running the client OS on emulated hardware. Facilities provided include cut and paste between the Client OS and the host system, much better display support with resizing, and even drag and drop of files (which I have not got working).

Up until about Ubuntu 14.04, the method for installing the VirtualBox Guest Additions was to install Ubuntu and then to (virtually) install the Guest Additions CD image provided by VirtualBox and run the installation from that. This no longer works under Ubuntu 17.10 (and I read that it didn’t work for Ubuntu 16, but I can’t verify that). Instead, packages are provided in Ubuntu itself to support the VirtualBox virtualisation facilities.

Based on a video by ‘Linux Video Tutorials’, here are the steps to install the VirtualBox Guest Additions in Ubuntu 17.10:

Settings for VirtualBox VM

    • set plenty of Video Memory (128Mb) in Display->Screen
    • set 3D Acceleration active in Display->Screen
    • enable PAE/NX under System->Processor
    • enable the bidirectional clipboard and drag and drop if you want them, in General->Advanced.

Installing the software

% sudo apt-get update
% sudo apt-get upgrade
% sudo apt-get install build-essential virtualbox-guest-dkms virtualbox-guest-x11
% sudo shutdown -r now

Checking that the software is working

The client OS should now boot with flexible display resizing and cut and paste support. The VirtualBox Guest Additions services should be enabled and running. I found I had to start them:

% hostnamectl
% sudo systemctl status virtualbox-guest-utils.service
% sudo systemctl start virtualbox-guest-utils.service
% sudo systemctl status virtualbox-guest-utils.service

% ps -ef | grep -i vbox
 root 280 2 0 11:36 ? 00:00:00 [iprt-VBoxWQueue]
 lemon 1501 1 0 11:37 ? 00:00:00 /usr/bin/VBoxClient --clipboard
 lemon 1504 1501 0 11:37 ? 00:00:00 /usr/bin/VBoxClient --clipboard
 lemon 1505 1 0 11:37 ? 00:00:00 /usr/bin/VBoxClient --display
 lemon 1506 1505 0 11:37 ? 00:00:00 /usr/bin/VBoxClient --display
 lemon 1512 1 0 11:37 ? 00:00:00 /usr/bin/VBoxClient --seamless
 lemon 1513 1512 0 11:37 ? 00:00:00 /usr/bin/VBoxClient --seamless
 lemon 1519 1 0 11:37 ? 00:00:00 /usr/bin/VBoxClient --draganddrop
 lemon 1520 1519 0 11:37 ? 00:00:00 /usr/bin/VBoxClient --draganddrop
 root 20751 1 0 11:40 ? 00:00:00 /usr/sbin/VBoxService

BT Internet uses the defunct SMTPS protocol for outgoing SMTP connections from a customer.  To set up Exim4 to be able to send mail via this interface requires a dirty hack.  The main part of the configuration is as described in this Ask Ubuntu article in the accepted answer (based on Tony Scelfo’s article).  However, that does not take account of BT’s peculiar configuration.  An additional step is required:  before running update-exim4.conf, edit /etc/exim4/exim4.conf.template. Find the section starting with

remote_smtp_smarthost:

Insert a new line following the line driver = smtp like this:

  protocol = smtps

Then continue with the steps in the article above. Use mail.btinternet.com::465 for the address of the smarthost when running dpkg-reconfigure.

This is a hack because it subverts the SMTP smarthost function to use SMTPS. However as you can only have one smarthost at a time, that doesn’t matter.

By the way, Raspbian on the Raspberry Pi uses Exim4 and this was how I got email working from it.

The instructions  for installing Emoncms on the Raspberry Pi are at available through the OpenEnergyMonitor site.  However I had to take some additional steps.  This page lists what I did, drawing heavily from those instructions with thanks.

I used a Raspberry Pi 2 model B.  This has Ethernet which I needed to connect to the network, though you could use a Wifi dongle.  Before you can use the Pi, you have to install its operating system on a micro-SD card–I used an 8Gbyte one.  To receive wireless data from emonTx and emonTh modules, you need an expansion card such as the RFM69Pi.

Install Raspbian

Using Windows, fetch the Raspbian installation image.  Extract the zip file to obtain the single image file, in my case called 2015-02-16-raspbian-wheezy.img.  Write this image to the micro-SD card using Win32DiskImager.  Insert the micro-SD card into the Raspberry Pi (it goes in upside down) and power up. Watch in amazement as the boot images whizz by on the HDMI-connected monitor.  The initial screens allow you to set the password for the “pi” user, and if you are not based in the UK, you can change the Internationalisation options.

The EmonCMS installation instructions suggest making an additional data partition on the SD card.  I preferred not to do this for two reasons: the data is precious and the SD card has limited life, even with the write buffering.  Instead, I plan to mount a shared folder from my Synology NAS drive to hold the data.  I cover this in a separate post, as it is tricky.  However, following the instructions caused the root filesystem to overflow during the installation of pre-requisite software, so I recommend expanding the filesystem to the full size of the SD card.  There’s an option to do this in the initial configuration screens.  If you’ve already exited, type “sudo raspi-config” to re-invoke the configuration screeen.

Disable Serial Port login

By default, Raspbian Wheezy is configured to support login on the serial port.  This would conflict with the use of that port for radio communications, so we need to disable logins on the serial port.  Edit the configuration files as follows:

sudo nano /etc/inittab

At the bottom of the file comment out the last line, by adding a ‘# ’ at the beginning:

# T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100

[Ctrl+X] then [y] then [Enter] to save and exit.  Next, edit the boot command line:

sudo nano /boot/cmdline.txt

Raspbian has been updated since the Emoncms instructions were written, and the line to change is now a little different.  Remove “console=ttyAMA0,115200” from the line, so that it reads

dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait

Change the filesystem to be mounted read-only

To reduce wear on the SD card, the filesystem can be mounted read-only, except when software updates are necessary.  The data will be stored on an external NAS drive. Before switching to read-only mode, we need to create commands to switch between read-only and read-write modes, and create a mount-point for the data partition. Do this while logged in as “pi”.

mkdir /home/pi/data

Turn on the TMPFS filesystem:

sudo cp /etc/default/tmpfs /etc/default/tmpfs.orig
sudo nano /etc/default/tmpfs

Find the line that says

#RAMTMP=no

Change it by removing the “# ” at the beginning of the line and changing “no” to “yes”:

RAMTMP=yes

Use [Ctrl+X] then [y] then [Enter] to save and exit.

Next, replace the filesystem mount point table:

sudo mv /etc/fstab /etc/fstab.orig
sudo sh -c "echo 'tmpfs           /tmp            tmpfs   nodev,nosuid,size=30M,mode=1777       0    0' >> /etc/fstab"
sudo sh -c "echo 'tmpfs           /var/log        tmpfs   nodev,nosuid,size=30M,mode=1777       0    0' >> /etc/fstab"
sudo sh -c "echo 'proc            /proc           proc    defaults                              0    0' >> /etc/fstab"
sudo sh -c "echo '/dev/mmcblk0p1  /boot           vfat    defaults                              0    2' >> /etc/fstab"
sudo sh -c "echo '/dev/mmcblk0p2  /               ext4    defaults,ro,noatime,errors=remount-ro 0    1' >> /etc/fstab"
sudo sh -c "echo '192.168.10.102:/volume1/emonpi2  /home/pi/data   nfs    user=pi                   0    0' >> /etc/fstab"
sudo sh -c "echo ' ' >> /etc/fstab"
sudo mv /etc/mtab /etc/mtab.orig
sudo ln -s /proc/self/mounts /etc/mtab

Note: The line referring to /home/pi/data will need to be changed to reflect your NAS drive’s IP address (or DNS name) and shared folder path.  See my separate post for instructions on making this work with Synology DiskStation DSM 5.1.

Create the commands to switch between read-only and read-write mode:

sudo nano /usr/bin/rpi-rw

In the empty file which loads, enter the following:

#!/bin/sh
sudo mount -o remount,rw /dev/mmcblk0p2  /
echo "Filesystem is unlocked - Write access enabled"
echo "Type 'rpi-ro' to lock"

[Ctrl+X] then [y] then [Enter] to save and exit, and then type the following to make the command executable:

sudo chmod +x  /usr/bin/rpi-rw

Similarly, create the opposite script:

sudo nano /usr/bin/rpi-ro

In the empty file which loads, enter the following:

#!/bin/sh
sudo mount -o remount,ro /dev/mmcblk0p2  /
echo "Filesystem is locked - Write access disabled"
echo "Type 'rpi-rw' to unlock"

[Ctrl+X] then [y] then [Enter] to save and exit, and then type the following to make the command executable:

sudo chmod +x  /usr/bin/rpi-ro

Reboot to bring the changes into effect:

sudo shutdown -r now

Log in again and change the permissions on the data directories. This may bring up error messages when using the Synology NAS.

sudo chmod -R a+w data
sudo chown -R pi data
sudo chgrp -R pi data

Continue to follow the instructions from “Installing Dependencies” in the instructions in installing Emoncms on the Raspberry Pi.  However, it turns out it is not really feasible to run MySQL with its data directory NFS mounted from Synology DSM.  This is due to a number of factors:

  • NFS mounts are not in place by the time MySQL starts (this could be worked around)
  • Synology NFS exports have weird permissions and ownership issues (see separate post; I have no solution)
  • I seem to recall criticism of the idea anyhow, on the basis that database storage should be local to the server for resiliency reasons.

All is not lost however.  Synology DSM is a flexible beast, and MySQL is available as a downloadable package for it, called MariaDB.

  1. Log into DSM as “admin” at http://diskstation:5000 (for example)
  2. Open “Package Center” and find MariaDB under “Utilities”. Press the “Install” button and wait until the magic is done.
  3. Find and install “phpMyAdmin”, also under “Utilities”.
  4. Open “phpMyAdmin” – this opens a new tab or browser window and presents the phpMyAdmin login screen.
  5. Log in as user “root” with no password.
  6. Under “General Settings”, select “Change password”.  In the pop-up window, enter a new password for the “root” MySQL user, and then press “Go”.
  7. In the ribbon at the top centre of the screen, select “Users”.
  8. Halfway down the screen, there’s a section called “New”.  Click on “Add user”.
  9. Under “Login information” enter the name “emoncms” in the “User name” field.
  10. Leave the “host” field set to the default value, to allow login from any host (or, for additional security, you could select “Use text field” and enter the IP address of the Raspberry Pi which will run the emoncms software).
  11. Enter a password for the “emoncms” user in the two fields provided.
  12. Under “Database for user”, select the checkbox marked “
  13. Click “Go” (which is at the very bottom right of the page).

Continue with the instructions, but disable the MySQL service on the Raspberry Pi, as we’ll use the one on the Diskstation instead:

rpi-rw
sudo update-rc.d mysql disable

In the section of the instructions entitled “Set emoncms database settings,” set the username to “emoncms” and the password to the password you set in step 11 above. Set the hostname to the IP address of the DiskStation.  Set the database name to “emoncms”.  Once these settings have been made, reboot the Raspberry Pi and point your web browser its newly installed EmonCMS installation.  Assuming the Raspberry Pi’s address is 192.168.1.22, this would be http://192.168.1.22/emoncms.  Register a new account on that page.  You should see an account status page.  In the “My Account” column, there is an entry called “Write API key”.  Copy that value (it’s a string of letters and numbers) as you’ll need it in the next step.

Enable writing to the SD card with the “sudo rpi-rw” command.  Edit the emonhub configuration file.  The instructions show how to edit the file, but they don’t explain what to change. Under “Reporters” you’ll see a line like this:

        apikey = xxxxxxxxxxxxxxxxxxxxxxxxxxxx

Change the string of “x”s to the value of the Write API key you copied above.  Save the file and exit the editor. Restart emonhub:

sudo service emonhub restart
rpi-ro

At this point your EmonCMS installation should be working (although a reboot of the Raspberry Pi might be a good idea) and upon logging in to EmonCMS and clicking on “Input”, you might be lucky enough to see any nodes you have already installed listed with their data values.

Three years ago, I wrote about making Trixbox work in the UK with TDM400P analogue line cards in a PC to connect to the PSTN and normal telephone extensions.  That setup served me well for ages, but I began to have difficulties with the Digium TDM400P cards when driving the lines to the exchange.  One of the channels started to play up, and as the DAHDI drivers are not really user-friendly either, I migrated to a pair of Linksys SPA3102 boxes.  These are sweet devices which have a lot more functionality in them than I use in my setup.  The SPA3102 provides an exchange line (PSTN) interface socket and also a traditional analogue phone interface socket, as well as two Ethernet interfaces, one intended to connect to the Internet (or in this case, to my Asterisk server) and one which I do not use, intended to serve a PC or home/office network.

Things in the Asterisk world have moved on in three years, and one change is that Trixbox is no longer supported.  Feeling that the time was right for an upgrade, I looked around the web and concluded that the front runner these days is PBX in a Flash.  This is very similar in scope to Trixbox, being a complete distribution presented as a bootable DVD which takes over and overwrites the entire machine it is booted on.  Beware!  For my upgrade, I disconnected the existing disk drive and used a different one for the new installation.  In typical calavier fashion, I didn’t write much down about my existing configuration and just went ahead and installed the new one.  I used PBX-in-a-Flash (PIAF) v2.0.6.2.4 which includes CentOS 6.2, and I used the 64-bit version.  PIAF requires the user to make quite a lot of choices during installation, and I chose FreePBX 2.9 and Asterisk 1.8, these being the most recent stable versions.

Installation takes forever,  belying its name.  The massive distribution doesn’t even include half the stuff you need, instead relying on the Internet to download and automatically install additional components.  There are several guides to installation on the Web, so I shan’t attempt one here.  What I do want to cover though is the mysteries of getting the TDM400P cards to be recognised.

The best guide to this which I found is Configuring and Testing Dahdi Hardware.  This covered the basics for me, though I had to recompile and reinstall the DAHDI drivers first:

# cd /usr/src/dahdi/
# make all
# make install
# make config

# cd tools
# make distclean
# ./configure
# make install

Then reboot the machine.

The undocumented magic for me was:

# dahdi_genconfig
# cd /etc/asterisk

Edit chan_dahdi.conf and add this line near the end, before chan_dahdi_additional.conf:

#include dahdi-channels.conf

After that, it’s time to reboot again and confirm that the channels exist:

# dahdi_cfg -vv

This should list lots of channels on the cards you are configuring.  They should also be visible in the Asterisk command-line interface (this output is specific to my setup):

piaf*CLI> dahdi show channels
Chan Extension  Context         Language   MOH Interpret        Blocked    State
pseudo            default                    default                         In Service
1            from-internal              default                         In Service
2            from-internal              default                         In Service
3            from-pstn                  default                         In Service
4            from-pstn                  default                         In Service
5            from-internal              default                         In Service
6            from-internal              default                         In Service
7            from-internal              default                         In Service
8            from-internal              default                         In Service
piaf*CLI>

One more magic step: use a browser to access the FreePBX administration interface on your server, and go to the Setup->DAHDi screen.  Under Analogue Hardware, find a line entitled “FXS Ports” and hit the “Edit” link.  If the Group entries are blank for your channels, enter “1” in each group field, then save and apply the settings.  At that point you should be able to create DAHDI extensions.  Hoorah!

The UK-specific changes I documented in my first article on this subject probably still apply, but I haven’t tried them as I no longer use the FXO ports.

Turn it on

{Administrator}=>service system modify
name = SNMP_AGENT
[state] = enabled
[log] = false
:service system modify name=SNMP_AGENT state=enabled log=disabled
{Administrator}=>

Add a RO community

{Administrator}=>snmp community add
securityname = ROCommunity
communityname = ******
Please retype communityname for verification.
communityname = ******
:snmp community add securityname=ROCommunity communityname=_DEV_E43BC822A4BAE52F
{Administrator}[snmp community]=>

Where the asterisks are type your chosen community name.

Once this is done I could access the router using SNMP version 1 and my RO community string.

This information was taken from http://usefulthings.org.uk/2007/01/be-24-meg-thompson-speedtouch-snmp-enable/ with thanks.

I was reading this interesting catalogue of telco-powered products which rely on the ultra-reliable batteries in telephone exchanges in many countries including the UK.  In my imagination, after the apocolypse, as we’re all sitting in our cellars drinking the best bottles of wine, but we can’t find the corkscrew because the lights have all gone off, we will rig up lights powered from the telephone line and everything will, somewhat briefly, be all right.

But then I had this fantastic idea.  Where I come from, there’s a trend to solar power and micro-generation with the government subsidising technologies and products which allow locally generated electricity to be sold into the national power grid, at really good prices too.  But why wait for the solar panels and micro-generation? We have a great source of local power already: the phone company!  I now plan to develop a gadget that takes power from the phone line and sells it to the power company.  OK, so the amount of power available will be very small, but it will always be there!

Another idea would be to take cheap energy in the night time, store it in hot water tanks, and generate electricity from it again in the day, selling it back to the power company at a better rate later.  Could this be my future income stream??

I’ve been trying to use Ruby libraries to access Sharepoint and am having trouble getting to the starting line.  NTLM authentication from Ruby doesn’t seem to be in a good state.  I’m using httpi-0.9.6 as my HTTP library, which by default uses httpclient-2.2.4, and for NTLM support I’m using httpi-ntlm-0.9.6.  All this is running under Ruby 1.8.7 which incorporates Net::NTLM 0.1.1 which http-ntlm relies on.

In my enterprise environment, I need to use NTLM authentication to access the Sharepoint server.  So, I format the username string supplied to httpi as “USERDOMAIN\username”.  It happens that the Sharepoint machine I want to access is in a different NTLM domain than the user account that I want to access it with.  When httpclient does the NTLM authentication phase, it uses facilities from Net::NTLM to format a Type 1 Message.  This is where the problems start.  Firstly, httpclient doesn’t do the right (undocumented) things necessary to get Net::NTLM to properly include the user’s domain into the request.

case state
  when :init
    t1 = Net::NTLM::Message::Type1.new
    t1.domain = domain if domain
    return t1.encode64
end

The trouble here is that this code doesn’t properly tell Net::NTLM that this domain information should be incorporated into the request.  As written, HTTPI generates a corrupt request, as can be seen from a Wireshark trace.  It’s additionally necessary to enable the resulting SecurityBuffer and to set a flag in the request indicating that the domain is being signalled:

case state
  when :init
    t1 = Net::NTLM::Message::Type1.new
    if domain
      t1.domain = domain
      t1.enable(:domain)
      t1.set_flag(:DOMAIN_SUPPLIED)
    end
    return t1.encode64

But when you do this, you find that the packet is still corrupt, because the offset to the domain name data in the resulting request is incorrect.  This is because of what appears to be a bug in Net::NTLM.  That’s not so easy to fix and play about with, because it’s incorporated into Ruby itself.  The problem is that the offsets are calculated by adding the sizes of the SecurityBuffers defined for the message type, regardless of whether a particular SecurityBuffer is active or not. For my testing purposes, I forced the additional “workstation” SecurityBuffer to be active (with an arbitrary non-null value) in the same way as the Domain, and by doing so, was able to generate a valid request.

But imagine my horror when my requests were still denied by the server.  Further investigation revealed that Net::NTLM ignores the user-specified domain when generating the subsequent Type 3 Message, substituting instead the Target value returned in the Challenge of the Type 2 Message.

Further hackery will be required to fix that, but another approach might be to use Curb instead of httpclient.

52 Brasstack Lane
Cleggithorpe
Yorkshire

3rd September

Dear Mary,

By heck, it were grand to go striding over the Peaks wi’ you last month.  I’ll never forget t’way t’rain trickled down your Parka.  And the lovely pink colour your face went when you choked on my Kendal Mint Cake.  And as for your blisters!  It fair makes me knees go weak to think about ’em.

Back here in Cleggithorpe, life’s just not the same.  At t’Miner’s Arms last night, Willy and Dick fair gave me a ribbin’.  “Eeee, Davey, lad,” they said, “hast lost thi sense o’ perspective?  She’s only a lass when all’s said and done.  Have another pint, lad, and we’ll get some black puddin’ on t’way home.”

But somehow, as I stared moodily at t’black puddin’, I couldn’t help thinkin’ o’you.  Tell me Mary, can you come up next weekend?  We could climb Cleggie Crags and hold hands in t’sleet.

Love,
Davey.