And now for something completely different

After almost 10 years in Master Data Management, most of which with the rather lovely view below, I’ve moved on to Watson.

I can’t quite believe I stayed in the same department that long but there were plenty of fresh challenges along the way, and no shortage of people inside and outside IBM to keep it interesting.

I’ve been particularly lucky to have had so much support building up the MDM Developers community, which should be in safe hands to continue growing in the future. (If you’re interested in MDM and haven’t attended one of the live tech talk sessions, I would definitely recommend trying one. There are recordings of all the previous events on YouTube and check out Dany’s OSGi talk for a great example.)

If my first day in Watson is anything to go by, the next challenge is going to be far from dull!

Advertisement

BigInsights Quicker Start

I’ve been taking a break from Liberty and JAX-RS recently to start tinkering with IBM’s BigInsights Hadoop distribution. To make things easier/more interesting my first attempts were using the Analytics for Hadoop service on BlueMix. In case it helps anyone, here’s what I ended up with before needing to install BigInsights myself:

And this is the script I used to upload data in the video (unfortunately I didn’t have any luck using the HttpFS API):

#!/bin/sh

BIUSER=biblumix
BIPASSWORD=password
BIURI=https://hostname:8443/data/controller/dfs

curl -iv –user ${BIUSER}:${BIPASSWORD} –insecure -X POST “${BIURI}/user/${BIUSER}/sample-data?isfile=false”

curl -iv –user ${BIUSER}:${BIPASSWORD} –insecure -X POST “${BIURI}/user/${BIUSER}/sample-data/orgdata.unl” –header “Content-Type:application/octet-stream” –header “Transfer-Encoding:chunked” -T “orgdata.unl”

curl -iv –user ${BIUSER}:${BIPASSWORD} –insecure -X POST “${BIURI}/user/${BIUSER}/sample-data/persondata.unl” –header “Content-Type:application/octet-stream” –header “Transfer-Encoding:chunked” -T “persondata.unl”

Notes:

• since recording the demo Bluemix has added a United Kingdom region, however it looks like the Analytics for Hadoop service is currently only available on the US South region.
• there is also now a BigInsights service on Bluemix which allows you to provision multi-node Hadoop clusters.

Java dumps

I recently had to debug a problem with the MDM Workbench where exporting a tailoring project for Information Server didn’t do anything. In fact it didn’t even report any problems!

Unfortunately the code in question likes to put a brave face on things and just reports that everything was OK, even when something goes wrong. This was the perfect opportunity to try out some of the diagnostic tools available for the IBM Java runtime, which I’ve been meaning to try for ages. I had an idea where the problem was likely to be but to find out for sure I started the workbench using the following command line:

eclipsec -vmargs -Xdump:system:events=catch,filter=java/lang/AbstractMethodError#com/ibm/mdm/tools/export/infoserver/job/MDMDatabaseDAO.queryDatabaseWithoutFilter

Sure enough the failing export produced a dump which I could check using the Memory Analyzer tool. You can get the IBM version via IBM Support Assistant but it’s probably easier to get the standard Eclipse Memory Analyzer and add the required IBM plugins from the DTFJ update site.

I’m fortunate enough to work in Hursley so I could pester someone who works on IBM Java runtime diagnostics, but there’s also a helpful article on developerWorks with details of how to trigger dumps, and how to run queries using OQL:

Debugging from dumps: Diagnose more than memory leaks with Memory Analyzer

So mystery solved- if you have an Oracle database and want to exporting tailoring projects for Information Server, make sure you set up the database connection with a more recent JDBC driver than the defaults.

Getting a handle on social MDM

Since this is the first work related post for a while, it’s probably a good idea to drop in the usual disclaimer as a reminder: “The postings on this site are my own and don’t necessarily represent IBM’s positions, strategies or opinions.”

Anyway, I recently spotted an MDM enhancement request, Improve Better support for social handle support, and it seemed odd that there wasn’t already something in the data model that could do a better job than using misc values. There are probably several options but I think this is what I’d do…

Add a new “Social Network” contact method category, and associated contact method types, for example: “Twitter”, “LinkedIn”, etc. Here’s what those look like in the Business Admin UI:

Now you can just add social network contact methods in the same way as you would for telephone numbers and email addresses, which means you get all the standard functionality you’re likely to need.

For example, here’s what an example getPerson response looks like with my Twitter and LinkedIn details:

<?xml version="1.0" encoding="UTF-8"?>
<TCRMService xmlns="http://www.ibm.com/mdm/schema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ibm.com/mdm/schema MDMDomains.xsd">
    <ResponseControl>
        <ResultCode>SUCCESS</ResultCode>
        <ServiceTime>17</ServiceTime>
        <DWLControl>
            <requesterName>mdmadmin</requesterName>
            <requesterLanguage>100</requesterLanguage>
            <requesterLocale>en</requesterLocale>
            <userRole>mdm_admin</userRole>
            <requesterTimeZone>EST5EDT</requesterTimeZone>
            <requestID>247353</requestID>
        </DWLControl>
    </ResponseControl>
    <TxResponse>
        <RequestType>getPerson</RequestType>
        <TxResult>
            <ResultCode>SUCCESS</ResultCode>
        </TxResult>
        <ResponseObject>
            <TCRMPersonBObj>
                <PartyId>531938348064117624</PartyId>
                <DisplayName>James Taylor</DisplayName>
                <PartyType>P</PartyType>
                <CreatedDate>2013-11-03 07:10:40.909</CreatedDate>
                <PartyLastUpdateDate>2013-11-03 07:10:41.175</PartyLastUpdateDate>
                <PartyLastUpdateUser>mdmadmin</PartyLastUpdateUser>
                <PartyLastUpdateTxId>153838348064091041</PartyLastUpdateTxId>
                <PersonPartyId>531938348064117624</PersonPartyId>
                <PartyActiveIndicator>Y</PartyActiveIndicator>
                <PersonLastUpdateDate>2013-11-03 07:10:41.767</PersonLastUpdateDate>
                <PersonLastUpdateUser>mdmadmin</PersonLastUpdateUser>
                <PersonLastUpdateTxId>153838348064091041</PersonLastUpdateTxId>
                <TCRMPartyAddressBObj>
                    <PartyAddressIdPK>537638348082796792</PartyAddressIdPK>
                    <PartyId>531938348064117624</PartyId>
                    <AddressId>539338348085784022</AddressId>
                    <AddressUsageType>3</AddressUsageType>
                    <AddressUsageValue>Business</AddressUsageValue>
                    <StartDate>2013-11-03 07:13:47.966</StartDate>
                    <PreferredAddressIndicator>Y</PreferredAddressIndicator>
                    <AddressGroupLastUpdateDate>2013-11-03 07:14:17.854</AddressGroupLastUpdateDate>
                    <AddressGroupLastUpdateUser>mdmadmin</AddressGroupLastUpdateUser>
                    <AddressGroupLastUpdateTxId>537038348085749779</AddressGroupLastUpdateTxId>
                    <LocationGroupLastUpdateDate>2013-11-03 07:14:17.839</LocationGroupLastUpdateDate>
                    <LocationGroupLastUpdateUser>mdmadmin</LocationGroupLastUpdateUser>
                    <LocationGroupLastUpdateTxId>537038348085749779</LocationGroupLastUpdateTxId>
                    <TCRMAddressBObj>
                        <AddressIdPK>539338348085784022</AddressIdPK>
                        <ResidenceType>11</ResidenceType>
                        <ResidenceValue>Office</ResidenceValue>
                        <AddressLineOne>IBM UK Ltd</AddressLineOne>
                        <AddressLineTwo>Hursley Park</AddressLineTwo>
                        <City>Winchester</City>
                        <ZipPostalCode>SO21 2JN</ZipPostalCode>
                        <CountryType>183</CountryType>
                        <CountryValue>Great Britain and N Ireland</CountryValue>
                        <AddressLastUpdateDate>2013-11-03 07:14:17.839</AddressLastUpdateDate>
                        <AddressLastUpdateUser>mdmadmin</AddressLastUpdateUser>
                        <AddressLastUpdateTxId>537038348085749779</AddressLastUpdateTxId>
                    </TCRMAddressBObj>
                </TCRMPartyAddressBObj>
                <TCRMPartyContactMethodBObj>
                    <PartyContactMethodIdPK>533238348104476375</PartyContactMethodIdPK>
                    <PartyId>531938348064117624</PartyId>
                    <ContactMethodId>534438348104476393</ContactMethodId>
                    <ContactMethodUsageType>10</ContactMethodUsageType>
                    <ContactMethodUsageValue>LinkedIn</ContactMethodUsageValue>
                    <SolicitationIndicator>N</SolicitationIndicator>
                    <StartDate>2013-11-03 07:17:24.762</StartDate>
                    <ContactMethodGroupLastUpdateDate>2013-11-03 07:17:24.778</ContactMethodGroupLastUpdateDate>
                    <ContactMethodGroupLastUpdateUser>mdmadmin</ContactMethodGroupLastUpdateUser>
                    <ContactMethodGroupLastUpdateTxId>535838348104476350</ContactMethodGroupLastUpdateTxId>
                    <LocationGroupLastUpdateDate>2013-11-03 07:17:24.762</LocationGroupLastUpdateDate>
                    <LocationGroupLastUpdateUser>mdmadmin</LocationGroupLastUpdateUser>
                    <LocationGroupLastUpdateTxId>535838348104476350</LocationGroupLastUpdateTxId>
                    <TCRMContactMethodBObj>
                        <ContactMethodIdPK>534438348104476393</ContactMethodIdPK>
                        <ReferenceNumber>http://www.linkedin.com/in/taylorjm</ReferenceNumber>
                        <ContactMethodType>3</ContactMethodType>
                        <ContactMethodValue>Social Network</ContactMethodValue>
                        <ContactMethodLastUpdateDate>2013-11-03 07:17:24.762</ContactMethodLastUpdateDate>
                        <ContactMethodLastUpdateUser>mdmadmin</ContactMethodLastUpdateUser>
                        <ContactMethodLastUpdateTxId>535838348104476350</ContactMethodLastUpdateTxId>
                    </TCRMContactMethodBObj>
                </TCRMPartyContactMethodBObj>
                <TCRMPartyContactMethodBObj>
                    <PartyContactMethodIdPK>539138348072352465</PartyContactMethodIdPK>
                    <PartyId>531938348064117624</PartyId>
                    <ContactMethodId>532838348072329035</ContactMethodId>
                    <ContactMethodUsageType>9</ContactMethodUsageType>
                    <ContactMethodUsageValue>Twitter</ContactMethodUsageValue>
                    <PreferredContactMethodIndicator>Y</PreferredContactMethodIndicator>
                    <StartDate>2013-11-03 07:12:03.523</StartDate>
                    <ContactMethodGroupLastUpdateDate>2013-11-03 07:12:03.57</ContactMethodGroupLastUpdateDate>
                    <ContactMethodGroupLastUpdateUser>mdmadmin</ContactMethodGroupLastUpdateUser>
                    <ContactMethodGroupLastUpdateTxId>536538348072325964</ContactMethodGroupLastUpdateTxId>
                    <LocationGroupLastUpdateDate>2013-11-03 07:12:03.523</LocationGroupLastUpdateDate>
                    <LocationGroupLastUpdateUser>mdmadmin</LocationGroupLastUpdateUser>
                    <LocationGroupLastUpdateTxId>536538348072325964</LocationGroupLastUpdateTxId>
                    <TCRMContactMethodBObj>
                        <ContactMethodIdPK>532838348072329035</ContactMethodIdPK>
                        <ReferenceNumber>@jtonline</ReferenceNumber>
                        <ContactMethodType>3</ContactMethodType>
                        <ContactMethodValue>Social Network</ContactMethodValue>
                        <ContactMethodLastUpdateDate>2013-11-03 07:12:03.289</ContactMethodLastUpdateDate>
                        <ContactMethodLastUpdateUser>mdmadmin</ContactMethodLastUpdateUser>
                        <ContactMethodLastUpdateTxId>536538348072325964</ContactMethodLastUpdateTxId>
                    </TCRMContactMethodBObj>
                </TCRMPartyContactMethodBObj>
                <TCRMPersonNameBObj>
                    <PersonNameIdPK>533538348064198718</PersonNameIdPK>
                    <NameUsageType>7</NameUsageType>
                    <NameUsageValue>Preferred</NameUsageValue>
                    <PrefixType>14</PrefixType>
                    <PrefixValue>Mr.</PrefixValue>
                    <GivenNameOne>James</GivenNameOne>
                    <StdGivenNameOne>JAMES</StdGivenNameOne>
                    <LastName>Taylor</LastName>
                    <StdLastName>TAYLOR</StdLastName>
                    <PersonPartyId>531938348064117624</PersonPartyId>
                    <StartDate>2013-11-03 07:10:41.986</StartDate>
                    <PersonNameLastUpdateDate>2013-11-03 07:10:41.986</PersonNameLastUpdateDate>
                    <PersonNameLastUpdateUser>mdmadmin</PersonNameLastUpdateUser>
                    <PersonNameLastUpdateTxId>153838348064091041</PersonNameLastUpdateTxId>
                    <LastUpdatedBy>mdmadmin</LastUpdatedBy>
                    <LastUpdatedDate>2013-11-03 07:10:41.986</LastUpdatedDate>
                </TCRMPersonNameBObj>
                <DWLStatus>
                    <Status>0</Status>
                </DWLStatus>
            </TCRMPersonBObj>
        </ResponseObject>
    </TxResponse>
</TCRMService>

Does that sounds sensible? Are there any enhancements? For example, I wonder about standardization: I put an ‘@’ on my Twitter ID, but I can easily imagine several variations ending up in there. I’ll leave that as an exercise for another day!

Check out the MDM Developers community for much more useful MDM related posts, forums and other resources.

Getting Technical: Resources for MDM Developers

One reason for the lack of new posts here is that I’ve been attempting to write my first article for the Mastering Data Management blog. After several false starts, and falling back to old school pencil and paper to finally get going, it’s finally done! So, for anyone interested in master data management, here is the very latest Mastering Data Management front page hot off the press:

Master Data Management links: March

It’s been a while since I last pulled together a few MDM related links, and I haven’t done it in March before, so here are a few sites I’ve been keeping an eye on lately. There’s a loose theme this month as well; these are a few of the MDM communities that are out there.

First up is a brand new group on LinkedIn. Created by Henrik at the end of February, this group already has 125 members and has immediately sparked some interesting discussions. (The group may be new but Henrik has been blogging for a while.)

• Multi-Domain MDM

There are plenty of other MDM groups on LinkedIn, and I’m a member of a few others, but so far none have really stood out. I’m hoping Multi-Domain MDM maintains momentum after it’s good start.

The next link is a community I’ve been a member of for a bit longer, in fact it featured in my first MDM links post! Dan was one of the earliest MDM bloggers I discovered, and is still posting on the Hub Designs Blog, so I’m not surprised this community is still going strong.

• The MDM Community

Blogging provided strong foundations for both these communities, and in some respects you don’t really need a purpose built ‘community’ site. An active blog with plenty of posts and discussions is just as good, and I’ve recently been enjoying this one:

• Mastering Data Management

And finally, communities are all about people, and that’s what twitter is good at. Lists make it easy to share people worth following and luckily Dan has one already, which saved me creating one myself!

• Dan Power’s List of People Interested in MDM

Are there any other MDM communities you’d recommend?

Happy New Year!

Following Dan Power and Crysta Anderson’s lead, I’m going to kick off the new year with a look back at the most popular posts from 2010. So with barely a pause and not even a drum roll, the winners are…

1. My second CurrentCost development board circuit

Way out ahead at number one is the only circuit board I’ve completed and put to regular use. Still working fine, apart from a brief pause when the batteries ran out. Kind of regretting replacing the batteries just in time for the recent spell of cold weather!

2. Master Information Hub: Getting Started

Not a close second, but still respectably ahead of the pack, this post is one I regularly point people to the first time they use the MDM Workbench. Hopefully it’s helped a few people out this year.

3. New clock radio

Leading the pack is this surprise entry to the top ten. Unlike some Joggler owners, I still use it fairly regularly and, apart from the occasional experiment, I’m still using the O2 software it came with. I did give Jolicloud another go yesterday, to see whether a little bluetooth keyboard helps; nice, but just not quite fast enough to switch permanently. Might give MeeGo a try next.

4. Get off my hashtag

Had a really interesting chat at the last homecamp about tagging, so this is a subject I’m likely to return to this year.

5. Weather Underground + Mashup Hub + Pachube = orb food

Maybe it’s just me but I get quite excited about the potential that this kind of data mashup has. Perhaps it’s because I’ve seen what you can do with enterprise data and software like Message Broker; now imagine the possibilities with open data and simple ways for anyone to manipulate it. (That’s not manipulation in the political sense of course!)

6. Master Information Hub: Social Bookmark Services

This follows on from the number 2 post, while the third in the series has some catching up to do and didn’t make the top 10. I also have some has some catching up to do; I hope to get to the next instalment early this year.

7. Liberal Democrats can’t win here

Politicians, gotta love ’em. I wonder how these graphs will look if we get proportional representation for the next election.

8. Home Easy ambient orb

All soldered together but not yet receiving that lovely data from the number 5 post. I’m currently pondering whether to just hard code things ‘for now’ or hack some more so that the three orbs could be programmed using the BlinkM sequencer.

9. Digital House Arrest

Politicians again. Really. Very. Annoying. I never did get a reply to my last letter to my MP, Chris Huhne.

10. Manifesto

Given that all politicians seem to be as bad as each other I was half tempted to stand as a RON (reopen nominations) candidate Anyone else up for a For The Win party next time?!

Highly commended: It takes two

Not actually in the top ten but this post about Hedge End twinning deserves an honourable mention for the great comments about Frome’s twins.

Happy new year!

New home for MDM Workbench on developerWorks

If you’ve visited any developerWorks spaces recently, you’ll have noticed there’s been a bit of a change; the old spaces application has been sunset. All is not lost though, and redirects are in place for the various new homes for old spaces, including the MDM Workbench space.

Instead of just migrating from spaces to a group on My developerWorks, I took the chance to open it up to a wider community of MDM Developers. While I do miss some of the features of the old spaces, one big advantage of the new group is that it’s much easier for anyone to participate.

Apart migrating information from the old MDM Workbench space, most of the activity in the group so far has been new people joining, which is a great start. I’m also looking forward seeing who the first person to add a new bookmark or feed will be. (I’m almost tempted to have another little competition to encourage people with a much prized Hursley postcard, just for a bit of fun!)

Anyway, if you’re a developer working with any of IBM’s MDM products, have a look at the new MDM Developers group, and let me know if you have any comments or suggestions.

Too busy to blog

I seem to have been far too busy to get any time for blogging lately. Actually, it’s the finishing off posts that I’ve been having trouble finding time for; I do still keep adding to the number of unpublished posts, with this selection boosting the number of drafts to 38:

• Master Information Hub: Social Bookmark Search Transaction
• My first Joggler app
• Election results!
• Icelandic Economy Bill
• dub dub dub dot fail
• Parking charge

Instead of finishing any of those off I’ve been…

…ordering a canvas for one of our wedding photos.

We’ve been married almost a year so it seemed like it would be a good idea to finally do something with some of our favourite photos!

The canvas has arrived, and looks great, but it’s still sealed in a back to keep all the dust off from…

…decorating the porch.

We have two front doors; one on the outside and one on the inside. I added the outer door not long after moving in to create small enclosed entrance hall/porch, which has been great for shoes, coats, assorted junk, and getting rid of anyone trying to sell something without letting out all the heat in the process. The old front door has needed a coat of paint ever since, but because I never look at the outside much I only painted the inside, until now. As a result, most of the house is currently covered in dust from all the sanding!

…playing with a new toy.

A bit of an impulse buy, but after gradually increasing the number of boxes I have plugged in all the time (ironic since it all started with the Current Cost meter!), I’ve cut down a few with a FRITZ!Box. Pretty happy with it so far: it’s a neat combination of ADSL wireless router, file/media server and DECT base station. Plus, thanks to a tip off from Jee Labs, @jthouse will soon be following @andy_house‘s lead and sending me an SMS if I miss any calls.

…not hacking the Joggler.

Apart from being in German, the FRITZ!Box media server works very well with the Joggler, so I’m still really happy to keep using the default Joggler software. For now that is: the Joggler appears to have been discontinued, and my question to O2 about creating Joggler apps seems to have vanished as well. I didn’t have much/any luck using Flash Develop, which is a shame because it seems to be a decent IDE, but Nick has been getting on pretty well on Linux. His Joggler music app looks very promising.

I have been keeping an eye on various Joggler hacks and the first one to have me seriously tempted, is Opera mobile running on the Joggler. I did briefly boot Ubuntu off a USB stick, but I don’t think it’s worth it just yet. On the other hand, Jolicloud does look like it might be ideally suited to running on the Joggler, unfortunately it doesn’t seem to work. I’m still hopeful that someone will get Jolicloud booting on the Joggler though; any ideas?

Actually, I think Sony have got a much better idea of how devices like the Joggler should work with the Dash. Getting something like that running on the Joggler would be awesome.

…thinking about home automation.

@lauracowen sparked off some great discussions on the Homecamp group, and I definitely like some of the ideas in Ian’s blog post. I had a chat to Laura on Friday, and the idea of a ‘boost’ is probably a good starting point. I’m sure that would help lower my gas bill, although I’m wondering whether an even lower tech solution might work with my existing heating controls. We often turn on the hot water to top it up, but forget to turn it off again. So, since there is currently no boost button, I’m tempted to just get a basic timer to sit next to the controller to set a reminder to come back and turn off the hot water after a short period. Or I guess I could even use the kitchen timer that’s already in the microwave.

…visiting London.

We spent yesterday in London, mainly to see The Woman in Black, which was excellent. It was hard to work out which of the blood-curdling screams were from the audience and which were part of the production! We also had some time to visit the National Gallery, which brought back a few memories of art lessons.

See how much things have changed since I ruined Turner’s Ulysses deriding Polyphemus with an oil rig?!

…watching Inception.

Really enjoyed Inception although, now I think about it, it’s riddled with plot holes that I never noticed at the time. Guess that’s pretty normal for dreams!

…blogging.

I’ve also been posting on the MDM Workbench blog, so there’ll be fewer work related posts on here now. (Still figuring out how to decide which blog to pick for MDM posts.)

MDM Workbench developerWorks blog

I’m very excited to announce that the MDM Workbench now has its own dedicated blog on developerWorks! If you’ve found any of my MDM Workbench posts useful, I would definitely recommend taking a look. Iain might have been the last person in my team to start blogging but he’s already posted two great articles on the new blog:

• Installing MDM Workbench
• Using JUnit to test Web Services Generated by the MDM Workbench

I’ll also be using the new blog to post MDM Workbench news which I was previously featuring on the MDM Workbench space.

That isn’t the end of MDM posts here either; I’ll still be writing about my own views of MDM and the workbench, and hopefully continuing my Social Bookmarking example.