Java dumps


I recently had to debug a problem with the MDM Workbench where exporting a tailoring project for Information Server didn’t do anything. In fact it didn’t even report any problems!

Unfortunately the code in question likes to put a brave face on things and just reports that everything was OK, even when something goes wrong. This was the perfect opportunity to try out some of the diagnostic tools available for the IBM Java runtime, which I’ve been meaning to try for ages. I had an idea where the problem was likely to be but to find out for sure I started the workbench using the following command line:

eclipsec -vmargs -Xdump:system:events=catch,filter=java/lang/AbstractMethodError#com/ibm/mdm/tools/export/infoserver/job/MDMDatabaseDAO.queryDatabaseWithoutFilter

Sure enough the failing export produced a dump which I could check using the Memory Analyzer tool. You can get the IBM version via IBM Support Assistant but it’s probably easier to get the standard Eclipse Memory Analyzer and add the required IBM plugins from the DTFJ update site.

I’m fortunate enough to work in Hursley so I could pester someone who works on IBM Java runtime diagnostics, but there’s also a helpful article on developerWorks with details of how to trigger dumps, and how to run queries using OQL:

Debugging from dumps: Diagnose more than memory leaks with Memory Analyzer

So mystery solved- if you have an Oracle database and want to exporting tailoring projects for Information Server, make sure you set up the database connection with a more recent JDBC driver than the defaults.

Certifiable


I recently switched to a new ISP, who have so far been excellent, however they use certificates signed by CAcert. While I generally agree with the principle behind that decision, it does make life difficult. They cheerfully say, “You can check the certificate is signed by CAcert, if you like, before accepting it.” But how?

Warning: the following approach to checking the certificate is signed by CAcert is quite likely to be rubbish, so it’s probably not a good idea to follow it! In my defense, it seemed like a reasonable balance between just accepting some random certificate and complete paranoia but if you know a better way, please let me know.

They aren’t on Windows but the CAcert root certificates are already included in various places, so it turns out that the simple answer might be to grab the certificate from a suitable Linux distribution. Just to be on the safe side, I wanted to find a distribution I could download securely. The best option I found was Tails, which has a secure download and, for extra peace of mind, can be verified with OpenPGP.

My chosen method for trusting the tails signing key was a tad more interesting on Windows due to the lack of an sha256sum command. Luckily it seems you can do anything in PowerShell, so with a little help from Brian Hartsock’s blog, this did the trick instead:

$ha = [System.Security.Cryptography.HashAlgorithm]::Create(“SHA256”)
$stream = New-Object System.IO.FileStream(“tails-signing.key”, [System.IO.FileMode]::Open, [System.IO.FileAccess]::Read)
$sb = New-Object System.Text.StringBuilder
$ha.ComputeHash($stream) | % { [void] $sb.Append($_.ToString(“x2”)) }
$sb.ToString()

 

All good, certificate verified. I would still rather Andrews & Arnold just used a proper certificate though: there are clearly problems with trusting all the certificate authorities that are included in browsers/operating systems by default but CAcert doesn’t exactly look like a fantastic example either, and normal users really don’t have any chance of making a more informed choice.

Getting a handle on social MDM


Since this is the first work related post for a while, it’s probably a good idea to drop in the usual disclaimer as a reminder: “The postings on this site are my own and don’t necessarily represent IBM’s positions, strategies or opinions.”

Anyway, I recently spotted an MDM enhancement request, Improve Better support for social handle support, and it seemed odd that there wasn’t already something in the data model that could do a better job than using misc values. There are probably several options but I think this is what I’d do…

Add a new “Social Network” contact method category, and associated contact method types, for example: “Twitter”, “LinkedIn”, etc. Here’s what those look like in the Business Admin UI:

cdcontmethcat

cdcontmethtp

Now you can just add social network contact methods in the same way as you would for telephone numbers and email addresses, which means you get all the standard functionality you’re likely to need.

For example, here’s what an example getPerson response looks like with my Twitter and LinkedIn details:

<?xml version="1.0" encoding="UTF-8"?>
<TCRMService xmlns="http://www.ibm.com/mdm/schema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ibm.com/mdm/schema MDMDomains.xsd">
    <ResponseControl>
        <ResultCode>SUCCESS</ResultCode>
        <ServiceTime>17</ServiceTime>
        <DWLControl>
            <requesterName>mdmadmin</requesterName>
            <requesterLanguage>100</requesterLanguage>
            <requesterLocale>en</requesterLocale>
            <userRole>mdm_admin</userRole>
            <requesterTimeZone>EST5EDT</requesterTimeZone>
            <requestID>247353</requestID>
        </DWLControl>
    </ResponseControl>
    <TxResponse>
        <RequestType>getPerson</RequestType>
        <TxResult>
            <ResultCode>SUCCESS</ResultCode>
        </TxResult>
        <ResponseObject>
            <TCRMPersonBObj>
                <PartyId>531938348064117624</PartyId>
                <DisplayName>James Taylor</DisplayName>
                <PartyType>P</PartyType>
                <CreatedDate>2013-11-03 07:10:40.909</CreatedDate>
                <PartyLastUpdateDate>2013-11-03 07:10:41.175</PartyLastUpdateDate>
                <PartyLastUpdateUser>mdmadmin</PartyLastUpdateUser>
                <PartyLastUpdateTxId>153838348064091041</PartyLastUpdateTxId>
                <PersonPartyId>531938348064117624</PersonPartyId>
                <PartyActiveIndicator>Y</PartyActiveIndicator>
                <PersonLastUpdateDate>2013-11-03 07:10:41.767</PersonLastUpdateDate>
                <PersonLastUpdateUser>mdmadmin</PersonLastUpdateUser>
                <PersonLastUpdateTxId>153838348064091041</PersonLastUpdateTxId>
                <TCRMPartyAddressBObj>
                    <PartyAddressIdPK>537638348082796792</PartyAddressIdPK>
                    <PartyId>531938348064117624</PartyId>
                    <AddressId>539338348085784022</AddressId>
                    <AddressUsageType>3</AddressUsageType>
                    <AddressUsageValue>Business</AddressUsageValue>
                    <StartDate>2013-11-03 07:13:47.966</StartDate>
                    <PreferredAddressIndicator>Y</PreferredAddressIndicator>
                    <AddressGroupLastUpdateDate>2013-11-03 07:14:17.854</AddressGroupLastUpdateDate>
                    <AddressGroupLastUpdateUser>mdmadmin</AddressGroupLastUpdateUser>
                    <AddressGroupLastUpdateTxId>537038348085749779</AddressGroupLastUpdateTxId>
                    <LocationGroupLastUpdateDate>2013-11-03 07:14:17.839</LocationGroupLastUpdateDate>
                    <LocationGroupLastUpdateUser>mdmadmin</LocationGroupLastUpdateUser>
                    <LocationGroupLastUpdateTxId>537038348085749779</LocationGroupLastUpdateTxId>
                    <TCRMAddressBObj>
                        <AddressIdPK>539338348085784022</AddressIdPK>
                        <ResidenceType>11</ResidenceType>
                        <ResidenceValue>Office</ResidenceValue>
                        <AddressLineOne>IBM UK Ltd</AddressLineOne>
                        <AddressLineTwo>Hursley Park</AddressLineTwo>
                        <City>Winchester</City>
                        <ZipPostalCode>SO21 2JN</ZipPostalCode>
                        <CountryType>183</CountryType>
                        <CountryValue>Great Britain and N Ireland</CountryValue>
                        <AddressLastUpdateDate>2013-11-03 07:14:17.839</AddressLastUpdateDate>
                        <AddressLastUpdateUser>mdmadmin</AddressLastUpdateUser>
                        <AddressLastUpdateTxId>537038348085749779</AddressLastUpdateTxId>
                    </TCRMAddressBObj>
                </TCRMPartyAddressBObj>
                <TCRMPartyContactMethodBObj>
                    <PartyContactMethodIdPK>533238348104476375</PartyContactMethodIdPK>
                    <PartyId>531938348064117624</PartyId>
                    <ContactMethodId>534438348104476393</ContactMethodId>
                    <ContactMethodUsageType>10</ContactMethodUsageType>
                    <ContactMethodUsageValue>LinkedIn</ContactMethodUsageValue>
                    <SolicitationIndicator>N</SolicitationIndicator>
                    <StartDate>2013-11-03 07:17:24.762</StartDate>
                    <ContactMethodGroupLastUpdateDate>2013-11-03 07:17:24.778</ContactMethodGroupLastUpdateDate>
                    <ContactMethodGroupLastUpdateUser>mdmadmin</ContactMethodGroupLastUpdateUser>
                    <ContactMethodGroupLastUpdateTxId>535838348104476350</ContactMethodGroupLastUpdateTxId>
                    <LocationGroupLastUpdateDate>2013-11-03 07:17:24.762</LocationGroupLastUpdateDate>
                    <LocationGroupLastUpdateUser>mdmadmin</LocationGroupLastUpdateUser>
                    <LocationGroupLastUpdateTxId>535838348104476350</LocationGroupLastUpdateTxId>
                    <TCRMContactMethodBObj>
                        <ContactMethodIdPK>534438348104476393</ContactMethodIdPK>
                        <ReferenceNumber>http://www.linkedin.com/in/taylorjm</ReferenceNumber>
                        <ContactMethodType>3</ContactMethodType>
                        <ContactMethodValue>Social Network</ContactMethodValue>
                        <ContactMethodLastUpdateDate>2013-11-03 07:17:24.762</ContactMethodLastUpdateDate>
                        <ContactMethodLastUpdateUser>mdmadmin</ContactMethodLastUpdateUser>
                        <ContactMethodLastUpdateTxId>535838348104476350</ContactMethodLastUpdateTxId>
                    </TCRMContactMethodBObj>
                </TCRMPartyContactMethodBObj>
                <TCRMPartyContactMethodBObj>
                    <PartyContactMethodIdPK>539138348072352465</PartyContactMethodIdPK>
                    <PartyId>531938348064117624</PartyId>
                    <ContactMethodId>532838348072329035</ContactMethodId>
                    <ContactMethodUsageType>9</ContactMethodUsageType>
                    <ContactMethodUsageValue>Twitter</ContactMethodUsageValue>
                    <PreferredContactMethodIndicator>Y</PreferredContactMethodIndicator>
                    <StartDate>2013-11-03 07:12:03.523</StartDate>
                    <ContactMethodGroupLastUpdateDate>2013-11-03 07:12:03.57</ContactMethodGroupLastUpdateDate>
                    <ContactMethodGroupLastUpdateUser>mdmadmin</ContactMethodGroupLastUpdateUser>
                    <ContactMethodGroupLastUpdateTxId>536538348072325964</ContactMethodGroupLastUpdateTxId>
                    <LocationGroupLastUpdateDate>2013-11-03 07:12:03.523</LocationGroupLastUpdateDate>
                    <LocationGroupLastUpdateUser>mdmadmin</LocationGroupLastUpdateUser>
                    <LocationGroupLastUpdateTxId>536538348072325964</LocationGroupLastUpdateTxId>
                    <TCRMContactMethodBObj>
                        <ContactMethodIdPK>532838348072329035</ContactMethodIdPK>
                        <ReferenceNumber>@jtonline</ReferenceNumber>
                        <ContactMethodType>3</ContactMethodType>
                        <ContactMethodValue>Social Network</ContactMethodValue>
                        <ContactMethodLastUpdateDate>2013-11-03 07:12:03.289</ContactMethodLastUpdateDate>
                        <ContactMethodLastUpdateUser>mdmadmin</ContactMethodLastUpdateUser>
                        <ContactMethodLastUpdateTxId>536538348072325964</ContactMethodLastUpdateTxId>
                    </TCRMContactMethodBObj>
                </TCRMPartyContactMethodBObj>
                <TCRMPersonNameBObj>
                    <PersonNameIdPK>533538348064198718</PersonNameIdPK>
                    <NameUsageType>7</NameUsageType>
                    <NameUsageValue>Preferred</NameUsageValue>
                    <PrefixType>14</PrefixType>
                    <PrefixValue>Mr.</PrefixValue>
                    <GivenNameOne>James</GivenNameOne>
                    <StdGivenNameOne>JAMES</StdGivenNameOne>
                    <LastName>Taylor</LastName>
                    <StdLastName>TAYLOR</StdLastName>
                    <PersonPartyId>531938348064117624</PersonPartyId>
                    <StartDate>2013-11-03 07:10:41.986</StartDate>
                    <PersonNameLastUpdateDate>2013-11-03 07:10:41.986</PersonNameLastUpdateDate>
                    <PersonNameLastUpdateUser>mdmadmin</PersonNameLastUpdateUser>
                    <PersonNameLastUpdateTxId>153838348064091041</PersonNameLastUpdateTxId>
                    <LastUpdatedBy>mdmadmin</LastUpdatedBy>
                    <LastUpdatedDate>2013-11-03 07:10:41.986</LastUpdatedDate>
                </TCRMPersonNameBObj>
                <DWLStatus>
                    <Status>0</Status>
                </DWLStatus>
            </TCRMPersonBObj>
        </ResponseObject>
    </TxResponse>
</TCRMService>

Does that sounds sensible? Are there any enhancements? For example, I wonder about standardization: I put an ‘@’ on my Twitter ID, but I can easily imagine several variations ending up in there. I’ll leave that as an exercise for another day!

Check out the MDM Developers community for much more useful MDM related posts, forums and other resources.

MQTT Joggler


Spurred on by the success of getting Mosquitto working on a Raspberry Pi, I recently had a play with MQTT on the Joggler. The O2 Joggler is still a great device for hacking and I currently have SqeezePlay OS running on it.

The reason I wanted to try and get MQTT on the Joggler was to make use of its light sensor, and publish light levels over MQTT. It all turned out to be pretty simple since most of the work has already been done by other people!

First thing to do was read the light sensor and get that working with an MQTT client. I had to skip some of Andy’s instructions and just built the client code rather than attempting to get doxygen working. Once I’d mashed up the light sensor code and publish example I could compile the worlds most pointless MQTT publisher:

gcc -Wall publightsensor.c -L../bin/linux_ia32 -I../src -lmqttv3c -lpthread -o publightsensor

Next it was time to check the results. This too was quick and easy thanks to the MQTT sandbox server, which has a handy HTTP bridge. And the final result… was a completely unscientific and slightly dingy light level 4! Now I’ll be able to turn on a lamp using an unreliable RF controlled socket and see whether it worked or not!

Update: the code really is all in the existing examples but I’ve created a Github Gist in case it’s any help: mqttjogglermashup.c (11 February 2013)

Building Mosquitto on a Raspberry Pi


Just a few notes in case anyone wants to build the latest version of Mosquitto on a Raspberry Pi before Roger makes it even easier. Luckily there were already a couple of articles describing how to build Mosquitto, and the comments definitely saved some head scratching:

Hopefully the following steps should get MQTT on your Raspberry Pi in double quick time…

Firstly install a few packages. The unscientific list I went with were these, since Python was definitely installed already:

$ sudo aptitude update
$ sudo aptitude install build-essential quilt libwrap0-dev libssl-dev devscripts python-setuptools

Next the sneaky tweak to avoid Python 2.6 errors while building. You can edit /usr/share/python/debian_defaults by hand to move the python2.6 entry from the list of supported versions, or the following should do the trick:

$ sudo cp /usr/share/python/debian_defaults /usr/share/python/debian_defaults.orig
$ sudo sed -E -e '/^old-versions|^unsupported-versions/ s/$/, python2.6/' -e '/^supported-versions/ s/python2\.6,+ +//' /usr/share/python/debian_defaults.orig | sudo tee /usr/share/python/debian_defaults

Now it’s time to grab all the Mosquitto source and packaging files needed to run the build. These were the latest at the time of writing:

$ mkdir mosquitto-build
$ cd mosquitto-build/
$ wget http://mosquitto.org/files/source/mosquitto-1.1.tar.gz -O mosquitto_1.1.orig.tar.gz
$ tar -xvf mosquitto_1.1.orig.tar.gz
$ wget http://mentors.debian.net/debian/pool/main/m/mosquitto/mosquitto_1.1-1.debian.tar.gz
$ tar -zxf mosquitto_1.1-1.debian.tar.gz -C mosquitto-1.1/

Hopefully everything should be ready to go, so kick off the build:

$ cd mosquitto-1.1/
$ debuild -us -uc

That’s all there is to it. Assuming everything worked, just install the packages:

$ sudo dpkg -i ../*mosquitto*.deb

Update: Alternatively, you could just use the new, experimental, debian repository for mosquitto. (13 January 2013)

Printing money


There’s a 3D printer in my future. Not so long ago it was a distant dream, with the cost of a 3D printer well beyond gadgety impulse buying limits, but costs are coming down and the future is approaching fast. I just need to find some way to justify the need for a 3D printer on the grounds of printing some replacement part essential for fixing the downstairs toilet! (Ok, so $500 for a Solidoodle is still a fair bit more than the cost of an old coat hanger.)

With all these cheap 3D printers on the way, we need more things to print. Speculating wildly, I can’t really see people wanting to print 3d photos. I’m sure existing CAD software is well suited to printing engineering orientated objects but that would be missing a huge opportunity.

A few years ago, I saw an incredible demo which Ann Marie Shillito gave to a few of us in Hursley. It was like being able to reach inside the screen and mould objects. The sense of touch added another dimension in a way which 3D displays just never have for me. I would definitely recommend trying to get a demo if at all possible.

In what should be perfect timing, Anarkik3D are raising money through IndieGoGo to fund the next stage of development for their Cloud9 software for 3D modelling. They have just 35 days left to reach their target of $120,000!

Photo © John Abella (CC BY 2.0)

Mad thermostat plan


Something I’ve really wanted to have a go at for a long time is hacking together a smarter heating system. The long process of moving house prevented any progress until now but I think a few things fell in to place today to get the project off the ground. And so a slightly mad thermostat plan was hatched…

The first part of the puzzle is a side effect of getting a solar water panel; to make the most of the solar panel we should only be using the boiler to top up the hot water at the end of the day. (Obviously that’s just theoretical at the moment because its pretty much been raining non stop since we got the solar panel!) Unfortunately the current central heating controller will only turn on the heating if the hot water is on at the same time, which is no help at all, so we really need a new controller to make the most of our zero carbon supply of hot water. There’s another, purely aesthetic reason to want a new heating controller; the kitchen upgrade got under way this week and the old controller has seen better days.

The current kitchen destruction has a bigger part to play though; now is an ideal opportunity to hide cables behind the new cupboards. For a while that didn’t actually seem like it was going to be all that much help, based on where the old thermostat was (hidden behind a door in the living room). I was looking at various programmable thermostats but the existing wiring from the thermostat restricted the options somewhat. The programmable thermostat we had in the old house seemed to work quite well with the existing wiring and controller… as long as the battery was fresh, otherwise it got confused about the temperature. Obviously not ideal for a thermostat, so I was hoping to avoid batteries this time!

Then, while being distracted by the wonky light switches yet again, inspiration struck…

The house hasn’t been constructed with the greatest care in the world, but those switches just could not have been original. The only thing that makes sense is if they were another botched DIY job, and it seemed highly unlikely that anyone would have dropped another cable run down the wall to do it. My hunch, based on the fact that there’s a water cylinder directly above those switches, is that there’s a horizontal cable run between the two. I checked, and… eureka! So now it’s a simple job to put both switches back on the same box, leaving an empty recessed box with a now bare kitchen wall behind it, making it perfect to run a new thermostat cable through the back of the box and round to the boiler! (Well I was pretty excited by this plan at the time.)

The thermostat to finish off this puzzle is a Heatmiser combined programmable thermostat and hot water timer. My theory is that I need the PRT/HW-N thermostat to go in the living room and a PRC powered relay card in place of the old central heating controller. I’m almost certain that the wiring will work with the existing system anyway, but if anyone has any experience/tips/gotchas, please let me know! That programmable thermostat should give me an RS485 interface to the thermostat which, if all goes well, won’t be too difficult to connect to my nanode– either with a bit of soldering, or one of these IO shields if I’m feeling lazy! The thing I like about this arrangement is that it should be possible to achieve plenty of automation if all goes well but, if there are any technical hitches, there’s a decent off the shelf controller to fall back on.

Update: a quick update since I’m doing some head scratching over whether the existing wiring from the central heating timer to the junction box in the airing cupboard will allow the heating to run independently from the hot water. If it does, the new thermostat is in place ready to go…

If it doesn’t, the new thermostat will just be a decorative feature while I figure out where I can sneak a new cable upstairs without disturbing the new kitchen! I don’t want to break the heating until I’m sure everything will work, so I’m working off a photo for now…

I’d love to hear from anyone who can decipher that lovely nest of wires! Here’s my theroy so far:

The black cable is the valve, and the other two cables that enter with it at the bottom are the pump and cylinder stat. It looks to me like the grey cable should be to turn the hot water off, which seems to be connected to the cylinder stat and a red wire from one of the cables above, which I’m hoping is from the timer. That just seems too easy for this house though, and I’m a bit puzzled by what the connections on the orange wire actually are. Lucky it’s all neatly connected and labelled so I can check the orange wire is connected to the cylinder stat and pump… bother. I guess I’m going to have to wait until Jo’s not looking so I can investigate more thoroughly!