In this post, I will share with you some of my experiences doing so called "environmental scanning" using free tools.
The examples will be more impressive if I actually showed some real examples, but I felt that while such tweets are public, I am still uncomfortable putting a spotlight on them, so I will not link to those tweets.
Your experience using the ideas here will vary depending on size of institution, penetration rate of Twitter etc .
For comparison purposes, NUS Libraries serves about 36K FTE (students and staff), and the most popular twitter account has over 500 followers. The official name is National University of Singapore libraries but is often abbreviated to just NUS Library or NUS libraries.
Many libraries now have Twitter account (more than 600 libraries), and use Twitter for diverse purposes including pushing out information to followers, as well as answering tweets sent directly to the account. But many librarians are also scanning for tweets about their libraries which are not directly address to them, with the aim of collecting compliments and doing service recovery for complaints.
Google alerts are a lot faster these days in picking up tweets (used to be a lag time of months), but in my experience it's still unreliable, in any case, why not just go to the source and use the built-in Twitter search.
Talking about Google alerts vs tweet alerts, one thing I noticed is that scanning for tweets has a really low false positive rate compared to Google alerts. This makes sense, in a 140 character tweet, space is at a premium, any keyword appearing is obviously not incidental.
By comparison google alerts pulls in lots of irrelevant hits, where say NUS and library is mentioned in passing on a long webpage.
After playing around with the Twitter search, you will notice that while it's pretty capable (you can use boolean operators), it doesn't do wildcards or truncation and unlike Google it's not smart enough to realise that "library" and "libraries" are different forms of the same root word.
Therefore tweets that say NUS libraries are not the same as tweets that say NUS library . So you should search for NUS libraries OR library . In fact, I recently realized that many shorten library even further to lib , so perhaps NUS libraries OR library OR lib is more to the point!
Other terms you might want to scan for include librarian for obvious reasons. You can also scan for full names e.g. National University of Singapore libraries OR library but as you might expect this usually gets no hits (remember the 140 character limit means abbreviations are the norm).
Andrew Burkhardt the emerging tech librarian at Champlain College in Burlington, VT (I'm sure his google alerts will pick this up! ), suggests we also scan for tweets with words, endnote, cite, research and need AND book OR article OR books OR articles .
I would add terms like database OR eresource OR journal OR catalogue (remember plurals are not automatically picked up) . If you have library branded names add those too. E.g we call our library catalogue LINC, our federated search is called Infogate
Of course it depends on your local situation, but so far for me, I find eresource and database being tweeted a few times (e.g complaining eresource is down, wondering what database to use for market reports), I've also gone with the broader book OR article OR books OR articles (without the word " need"). There are a few false hits (people happy about getting paper published) but its not overwhelming.
Incidentally, I had to split these searches into several different search queries rather than combining them all into one, because the maximum length of any twitter search is 140 characters,
Tweet with geolocation scanning
Tweet with scanning of users in a list
I find quite a few tweets talking about the library (or libraries or lib) by users but aren't picked up as they don't mention NUS , are tweeted outside NUS, or are not geotagged.
I won't share much here, mainly because my approach is getting a lot of false hits, so I'm trying to figure this part out. I will add though Google alerts have one huge weakness, while they recently started to index Facebook pages (this is so new that a few days ago, a google search restricted to site:facebook.com gave zero results!), but not public status updates!
Given the recent moves by Facebook to make status updates public by default, this means Google misses a big bunch of information! But is there a Bing alert? Turns out it didn't.
But I found this hidden trick, that allows you to convert any Bing search to a RSS feed. Simply do the Bing search you want and then add &format=rss to the end of the url!
You only want facebook results, so do a bing search site:www.facebook.com nus library , grab the url, and add &format=rss to the end of the url
Scanning of specific social media services - Plurk , Friendster etc
A somewhat minor trick is to use the social search engine Socialmention . This is a very powerful search engine that indexes most of the more popular and obscure social media services, and more importantly allows you to restrict searches to specific social media services, and generate a RSS feed from the search.
I basically use this to monitor Plurk and Friendster , which is fairly popular here.
Real time or near real-time notifications
So far, I talked about doing different searches and all the method allow you to generate RSS feeds directly (except http://listimonkey.com/ which is email only), but how do you put them all together?
So far, greatest successes at environment scanning came from users who were amazed at the speed of response. One user raved to his followers (who in term Retweeted to theirs, estimated reach of 3,000 ), about how we picked up on his ranting and within minutes fixed the problem.
This was possible only with real-time scanning via a Twitter client. I personally use TweetDeck which pops up alerts very quickly. The main problem here is that for some reason , Tweetdeck doesn't support searches that take into account locations. I tried a couple of popular Twitter clients, both web-based and desktop, and almost all had the same problem.
Anyone know of any twitter client that supports alerts of twitter searches with locations?
RSS feeds these days can be much faster with support of pubsubhubbub (Google reader supports this)
or SUP (Friendfeed) , so maybe it isn't too bad.
There are many RSS feed readers out there, but I think GoogleReader and Friendfeed are both good choices, they allow you to store everything in one searchable place and both have mechanisms in place that supposedly allow "instant" RSS updates
GoogleReader has the advantage of greater support and a more certain future than Friendfeed, where a recent sale to Facebook has created lots of doubts of it's future.
On the other hand, Googlereader doesn't yet have a build-in service that automatically popups to alert you that a feed was updated , though no doubt addons exist that do this.
Here's how I do it.
First, I use Tweetdeck to pick up the normal tweets that aren't based on location, this is very fast within minutes (adjust the
In addition, I setup a private Friendfeed group, toss all the RSS feeds into it including
- Tweets based on location, Tweets with institution name
- Google alerts in RSS feed
- Bing searches in RSS feed
- Social media in RSS feed
Overcomplicated? Maybe. But since doing this, I realize that our users are talking about libraries a lot (both positive and negative), sometimes a lot more than a simple google alert, or twitter search would indicate.
But is such scanning going too far? Sure the tweets and information are all public, but would users object if they knew libraries were doing such things? My impression is many users have a "there is so much stuff on the net, they won't notice me" mentality.... right until they realize organizations are doing such scans and searches.
In any case, the aim of libraries in doing this is to be able to help and not to police users, so any users reading this should not be alarmed!
Thus far, twitter has helped to quickly detect problems (systems down) or enable librarians to proactively form help when needed (database needed for assignment) and seems to be worth the investment of time. What do you guys think?