twitter and some api

Although early networks involved human-machine interaction,But today's networks have involved machine-to-machine interactions,This interaction is supported using web services.Most popular websites have such services-from a variety of google services to linkedin, facebook and twitter. Through APIs created by web services, external applications can query or manipulate the content on the website.

Web services can be implemented in a number of ways.One of the most popular practices today is representational state transfe (rest). One implementation of rest is through the well-known http protocol, allowing http to exist as a medium of restful architecture (using standard http operations such as get, put, post, delete). Twitter's API is developed as an abstraction on top of this medium.In this approach,No knowledge of data formats such as rest, http or xml or json,Instead, it is an object-based interface that is clearly integrated into the Ruby language.

A quick showcase of ruby ​​and twitter

Let's explore how to use twitter api in ruby. First, we need to get the resources we need,If you are using ubuntu linux®like me, use the apt framework.

To get the latest full Ruby distribution (about 13mb downloads), use this command line:

$sudo apt-get install ruby1.9.1-full

Then use the gem utility to grab the twitter gem:

$sudo gem install twitter

Now that you have everything you need for this step,We continue,Test the twitter wrapper.This example uses a shell called an interactive ruby ​​shell (irb),This shell allows real-time execution of ruby ​​commands and experiments using languages.irb has a lot of features,But we only use it for some simple experiments.

Listing 1 shows a conversation with irb,The conversation was divided into three sections for easy reading.The first paragraph (lines 001 and 002) prepares the environment by importing the necessary runtime elements (the require method loads and executes the specified library). The next paragraph (line 003) illustrates the use of the twitter gem to display the latest tweets from ibm®developerworks® ;. As shown,Use the user_timeline method of the client ::timeline module to display a message,This first example illustrates the functionality of Ruby's "chain method". The user_timeline method returns an array of 20 messages.Then link into the method first, this is to extract the first message from the array (first is a method of the array class), and then extract the text field from this messagePut it in the output by the puts method.

The next paragraph (line 004) uses a user-defined location field,This is an open-ended field.Users can provide useful or useless location information.In this example,The user module captures the user information defined by the location field.

The last paragraph (starting from line 005) studied the twitter ::search module, which provides an extremely rich interface for searching twitter. In this example,The first is to create a search instance (line 005), then specify a search on line 006,Search for a message recently sent by the user lulzsec that contains the word why,The results list has been truncated and edited.Search settings will always be there,Because the search instance maintains the defined filters,You can clear these filters by executing search.clear.

Listing 1. Experimenting with the twitter api via irb

irb (main):001:0>require "rubygems"
irb (main):002:0>require "twitter"
irb (main):003:0>puts twitter.user_timeline ("developerworks"). first.text
dw twitter is saving #ibm over $600k per month:will #google + add to that?>
http://t.co/hirwir7 #tech #webdesign #socialmedia #webapp #app
irb (main):004:0>puts twitter.user ("mtimjones"). location
colorado, usa
irb (main):005:0>search=twitter ::search.new
=>#<twitter ::search:[email protected]_token_secret=nil,@ endpoint="https://api.twitter.com/1/",@ user_agent="twitter ruby ​​gem 1.6.0",@ oauth_token=nil,@consumer_secret=nil,@ search_endpoint="https://search.twitter.com/",@query={:tude =>[],:q =>[]},@cache=nil,@gateway=nil,@consumer_key=nil,@ proxy=nil,@format =:json,@adapter =:net_http<
irb (main):006:0>search.containing ("why"). to ("lulzsec").
result_type ("recent"). each do | r | puts r.text end
@lulzsec why not stop posting<bleep>and get a full time job! mysqli isn "t
hacking you<bleep> ;.
irb (main):007:0>

Next, let ’s take a look at the user pattern in twitter.You can also do this with irb,But I rearranged the results,In order to simplify the description of the internal structure of twitter users.Listing 2 shows the output of the user structure,This is a hashie ::mash in ruby. This structure is useful,Because it allows the object to have a hash key accessor (public object) of class methods. As you can see from Listing 2,This object contains a wealth of information (user-specific and rendered information), including the current user status (with geocoding information). A tweet also contains a lot of information,You can use the user_timeline class to easily generate this information visually.

Listing 2. Analytic structure for twitter users (ruby perspective)

irb (main):007:0>puts twitter.user ("mtimjones")
<#hashie ::mash
created_at="wed oct 08 20:40:53 +0000 2008"
default_profile=false default_profile_image=false
description="platform architect and author (linux, embedded, networking, ai)."
id=16655901 id_str="16655901"
location="colorado, usa"
name="m. tim jones"
profile_sidebar_border_color="181a1e" profile_sidebar_fill_color="252429"
status =&#hashie ::mash
contributors=nil coordinates=nil
created_at="sat jul 02 02:03:24 +0000 2011"
id=86978247602094080 id_str="86978247602094080"
in_reply_to_status_id=nil in_reply_to_status_id_str=nil
in_reply_to_user_id=225663702 in_reply_to_user_id_str="225663702"
place =&#;#hashie ::mash
attributes =&#hashie ::mash>
bounding_box =&#;hashie ::mash
coordinates=[[[-105.178387, 40.12596],[-105.034397, 40.12596],[-105.034397, 40.203495],[-105.178387, 40.203495]]]
country="united states" country_code="us"
full_name="longmont, co"
name="longmont" place_type="city"
text="@ anonymousirc @anonymousabu @lulzsec @atopiary @anonakomis practical reading
for future reference ... lulz \ "prison 101 \" http://t.co/sf8jih9 "truncated=false
time_zone="mountain time (us&canada)"
irb (main):008:0>

This is the quick part of the presentation.Now let ’s study some simple scripts,You can use ruby ​​and twitter api in these scripts to collect and visualize data.in this process,You will understand some concepts of twitter,Examples include authentication and frequency restrictions.

Mining twitter data

The next few sections introduce several scripts that use twitter api to collect and present available data.These scripts focus on their simplicity,But you can create new features by extending and combining them.In addition, this section will also mention the twitter gem api, which has more available functions.

It ’s important to note thatWithin the specified time,twitter api only allows customers to make a limited number of calls,This is also Twitter's frequency limit request (now it does not exceed 150 times an hour), which means that after a certain number of uses,You will get an error message,And ask you to wait for a while before submitting a new request.

User Info

Recall the vast amount of information available for each twitter user in Listing 2,This information is only accessible if the user is not protected.Let's take a look at how to extract user information and present it in a more convenient way.

Listing 3 shows a simple ruby ​​script (based on the user interface display name) to retrieve user information, and then display some more useful content,Use the ruby ​​method to_s to convert the value to a string when needed.have to be aware of is,First of all users are unprotected,Otherwise, her/his data cannot be accessed.

Listing 3. Simple script to extract twitter user data (user.rb)

#!/usr/bin/env ruby
require "rubygems"
require "twitter"
screen_name=string.new argv [0]
a_user=twitter.user (screen_name)
if a_user.protected!=true
puts "username:" + a_user.screen_name.to_s
puts "name:" + a_user.name
puts "id:" + a_user.id_str
puts "location:" + a_user.location
puts "user since:" + a_user.created_at.to_s
puts "bio:" + a_user.description.to_s
puts "followers:" + a_user.followers_count.to_s
puts "friends:" + a_user.friends_count.to_s
puts "listed cnt:" + a_user.listed_count.to_s
puts "tweet cnt:" + a_user.statuses_count.to_s
puts "geocoded:" + a_user.geo_enabled.to_s
puts "language:" + a_user.lang
puts "url:" + a_user.url.to_s
puts "time zone:" + a_user.time_zone
puts "verified:" + a_user.verified.to_s
tweet=twitter.user_timeline (screen_name) .first
puts "tweet time:" + tweet.created_at
puts "tweet id:" + tweet.id.to_s
puts "tweet text:" + tweet.text

To call this script,You need to make sure it is executable (chmod + x user.rb) and call it with the name of a user.Listing 4 shows the results of calling with user developerworks,The user's information and current status are given (the last tweet). The thing to note here is thatTwitter defines people who follow you as followers;and people you follow as friends.

Listing 4. Example output from user.rb

$./user.rb developerworks
user since:fri sep 19 13:10:39 +0000 2008
bio:ibm "s premier web site for java, android, linux, open source, php, social,cloud computing, google, jquery, and web developer educational resources
listed cnt:3801
tweet cnt:9831
time zone:pacific time (us&canada)
tweet time:sun jul 17 01:04:46 +0000 2011
tweet id:92399309022167040
tweet text:dw twitter is saving #ibm over $600k per month:will #google + add to that?>
http://t.co/hirwir7 #tech #webdesign #socialmedia #webapp #app

Friends' popularity

Research your friends (people you follow) and collect data to understand their popularity.In this example,Collect data from friends and sort by their number of followers,This simple script is shown in Listing 5.

In this script,After knowing the users i want to analyze (based on the interface display name),Create a user hash table,A ruby ​​hash (or an associated array) is a data structure that allows you to define storage keys instead of simple numeric indexes.Then, index this hash table by twitter's interface name,The correlation value is the number of users' followers.This process simply traverses your friends and puts the number of their followers in a hash table,Then (in descending order) the hash table,Then put it in the output.

Listing 5. Script about friends' popularity (friends.rb)

#!/usr/bin/env ruby
require "rubygems"
require "twitter"
name=string.new argv [0]
#iterate friends, hash their followers
twitter.friends (name) .users.each do | f |
#only iterate if we can see their followers
if (f.protected.to_s!="true")
user [f.screen_name.to_s]=f.followers_count
user.sort_by {| k, v | -v} .each {| user, count | puts "#{user}, #{count}"}

An example output of the friend script in Listing 5 is shown in Listing 6. I trimmed the output to save space,But as you can see,In my direct network,readwriteweb (rww) and playstation are very popular twitter users.

Listing 6. Interface output from the friend script in Listing 5

$./friends.rb mtimjones
rww, 1096862
playstation, 1026634
harvardbiz, 541139
tedtalks, 526886
lifehacker, 146162
wandfc, 121683
anonymousirc, 117896
itunespodcasts, 82581
adultswim, 76188
forrester, 72945
googleresearch, 66318
gartner_inc, 57468
developerworks, 48518

Where are my fans from

Recall from Listing 2 that Twitter provides a wealth of location information,There is an open-ended, user-defined location field with optional geocoded data.However, user-set time zones can also provide clues to the actual location of fans.

In this example,You are building a mashup application to extract time zone data from twitter fans,Then use google charts to visualize this data.google charts is a very interesting project,It allows you to build a variety of different types of charts over the network;Define the chart type and data as an http request and render it directly in the browser as the result of the response.To install the ruby ​​gem for google charts, use the following command line:

$gem install gchartrb

Following a similar pattern,The script accepts an interface name,Then iterate over the user's followers,Extract the time zone from the current fan and store it in the tweetlocation hash table.have to be aware of is,You first need to check whether the key value already exists in the hash table,If yes,Increment the count of this key value.You can also remember the number of all time zones,To be used for subsequent percentage calculations.

The last part of this script is to construct the url of the google pie chart, create a new piechart, specify some options (size, title, whether it is 3d, etc.);then, iterate through the time zone hash table,Find the data for the time zone string for the chart (deleted ampersands) and the percentage of the time zone count to the total.

Listing 7. Building a pie chart with twitter's time zone (followers-location.rb)

#!/usr/bin/env ruby
require "rubygems"
require "twitter"
require "google_chart"
screen_name=string.new argv [0]
twitter.configure do | config |
#iterate followers, hash their location
followers=twitter.followers.users.each do | f |
if (loc.length>0)
if tweetlocation.has_key?(loc)
tweetlocation [loc]=tweetlocation [loc] + 1
tweetlocation [loc]=1
timezones=timezones + 1.0
#create a pie chart
googlechart ::piechart.new ("650x350", "time zones", false) do | pc |
tweetlocation.each do | loc, count |
pc.data loc.to_s.delete ("&"), (count/timezones * 100) .round
puts pc.to_url

To execute the script in Listing 7,Give it a twitter interface display name,Then copy and paste the resulting URL into your browser.Listing 8 shows this process and the resulting url.

Listing 8. Invoking the followers-location script (results come in only one line)

$./followers-location.rb mtimjones
http://chart.apis.google.com/chart?chl=seoul|santiago|paris|mountain+time+ (us ++ canada) |
madrid | central + time + (us ++ canada) | warsaw | kolkata | london | pacific+ time + (us ++ canada) |
new + delhi | pretoria | quito | dublin | moscow | istanbul | taipei | casablanca | hawaii | mumbai |
international + date + line + west | tokyo | ulaan + bataar | vienna | osaka | alaska | chennai | bern |
brasilia | eastern + time + (us ++ canada) | rome | perth | la + paz
&Chs=650x350&chtt=time + zones&chd=s:kddykckdockdkdddddkddkddddokk9ddd&cht=p

Paste the URL from Listing 8 into your browser,You can get the results shown in Figure 1.

Figure 1. Pie chart of twitter fan position distribution

Twitter user behavior

twitter contains a lot of data,You can mine certain data to understand certain elements of user behavior.Two simple examples will be used to analyze when Twitter users publish messages and through what applicationsYou can use the following two simple scripts to extract and visualize this information.

The script given in Listing 9 traverses a specific user's tweet (using the user_timeline method), and then extracts the specific time when the tweet was formed from each tweet.A simple hash table is used again to count the counts of each day of the week.Then use google charts to generate a bar chart similar to the previous time zone example.

Listing 9. Building a histogram of tweet posting times (tweet-days.rb)

#!/usr/bin/env ruby
require "rubygems"
require "twitter"
require "google_chart"
screen_name=string.new argv [0]
timeline=twitter.user_timeline (screen_name,:count =>200)
timeline.each do | t |
tweetday=t.created_at.to_s [0..2]
if dayhash.has_key?(tweetday)
dayhash [tweetday]=dayhash [tweetday] + 1
dayhash [tweetday]=1
googlechart ::barchart.new ("300x200", screen_name,:vertical, false) do | bc |
bc.data "sunday", [dayhash ["sun"]], "00000f"
bc.data "monday", [dayhash ["mon"]], "0000ff"
bc.data "tuesday", [dayhash ["tue"]], "00ff00"
bc.data "wednesday", [dayhash ["wed"]], "00ffff"
bc.data "thursday", [dayhash ["thu"]], "ff0000"
bc.data "friday", [dayhash ["fri"]], "ff00ff"
bc.data "saturday", [dayhash ["sat"]], "ffff00"
puts bc.to_url

Figure 2 provides the results of using the developerworks account to execute the tweet-days script in Listing 9,As shown,Wednesday is the most active tweet.The least active are Saturday and Sunday.

Figure 2. Histogram comparing daily tweet activity

The next script determines from which source a particular user tweets. You can publish a message in several ways.This script does not code every one.As shown in Listing 10, a similar pattern is used to extract content on the timeline for a given user,Then try to decode the origin of the tweet message in the hash table.Later this hash table will be used to create a simple pie chart,Use Google charts to visualize the data.

Listing 10. Pie chart for building a user's tweet source (tweet-source.rb)

#!/usr/bin/env ruby
require "rubygems"
require "twitter"
require "google_chart"
screen_name=string.new argv [0]
timeline=twitter.user_timeline (screen_name,:count =>200)
timeline.each do | t |
if (t.source.rindex ("blackberry")) then
elsif (t.source.rindex ("snaptu")) then
elsif (t.source.rindex ("tweetmeme")) then
elsif (t.source.rindex ("android")) then
elsif (t.source.rindex ("linkedin")) then
elsif (t.source.rindex ("twitterfeed")) then
elsif (t.source.rindex ("twitter.com")) then
if tweetsource.has_key?(src)
tweetsource [src]=tweetsource [src] + 1
tweetsource [src]=1
googlechart ::piechart.new ("320x200", "tweet source", false) do | pc |
tweetsource.each do | source, count |
pc.data source.to_s, count
puts "\ npie chart"
puts pc.to_url

Figure 3 provides a visual chart,Shows a set of tweets for Twitter users who are interested in using,The traditional twitter website is most commonly used,Followed by mobile phone applications.

Figure 3. Pie chart of the tweet source of the twitter user

Graphs reflecting fans

twitter is a huge network of users,These users form a network diagram.As you can see through the script,It's easy to walk through your contacts,It's also easy to walk through their contacts.Even at this level,This has also established the fundamentals of a big picture.

To visualize graphics,I chose to use the graph visualization software graphviz. On ubuntu, use the following command line,You can easily install this tool:

$sudo apt-get install graphviz

The script in Listing 11 goes through the user's followers,Then go through these fans their fans.The only real difference in this model is the construction of the dot format file of graphviz,graphviz uses a simple script format to define graphs,These graphics appear as part of the twitter users you listed.As the list shows,Graphs can be defined simply by specifying the relationships of the nodes.

Listing 11. Visualizing Twitter fan graph (followers-graph.rb)

#!/usr/bin/env ruby
require "rubygems"
require "twitter"
require "google_chart"
screen_name=string.new argv [0]
twitter.configure do | config |
my_file=file.new ("graph.dot", "w")
my_file.puts "graph followers {"
my_file.puts "node [fontname=arial, fontsize=6];"
#iterate followers, hash their location
followers=twitter.followers (screen_name,:count =>10) .users.each do | f |
#only iterate if we can see their followers
if (f.protected.to_s!="true")
my_file.puts "\" "+ screen_name +" \ "-\" "+ f.screen_name.to_s +" \ ""
followers2=twitter.followers (f.screen_name,:count =>10) .users.each do | f2 |
my_file.puts "\" "+ f.screen_name.to_s +" \ "-\" "+
f2.screen_name.to_s + "\" "
my_file.puts "}"

Run the script in Listing 11 on a user, put the results in a dot file, and then use graphviz to generate the image.First call the ruby ​​script to collect the graph data (stored as graph.dot);then use graphviz to generate the image (circo is used here, the tool specifies a circular layout). The process of generating an image is defined as follows:

$./followers-graph.rb mtimjones
$circo graph.dot -tpng -o graph.png

The resulting image is shown in Figure 4. have to be aware of is,Since this twitter chart can be huge,Because we limit the size of the chart by minimizing the number of users and their followers to list (through the:count option in Listing 11).

Figure 4. Example of twitter fan graph (subset of extreme cases)

location information

With the feature enabled,Twitter collects geographic data about you and your tweets.This data, consisting of longitude and latitude information, can be used to pinpoint the user's location or where the tweet was sent from.In addition, combining search with these messages,You can identify some places or some people based on the defined location or where you are.

Not all users or tweets are geo-enabled (for privacy reasons), but this information is a very attractive aspect of any twitter experience.Let's look at a script,This script allows you to visualize geolocation data,There is another script that allows you to search using this data.

The first script (shown in Listing 12) grabs the user's longitude and latitude data (recall the bounding box in Listing 2), although the bounding box is a polygon that defines a user area,But I simplified it,Only one point in this area was used.With this data,I generated a simple javascript function in a simple hmtl file. This javascript code interfaces with google maps and gives a top-down map of this location (given the longitude and latitude data extracted from twitter users).

Listing 12. A ruby ​​script to construct a user map (where-am-i.rb)

#!/usr/bin/env ruby
require "rubygems"
require "twitter"
twitter.configure do | config |
screen_name=string.new argv [0]
a_user=twitter.user (screen_name)
if a_user.geo_enabled == true
long=a_user.status.place.bounding_box.coordinates [0] [0] [0];
lat=a_user.status.place.bounding_box.coordinates [0] [0] [1];
my_file=file.new ("test.html", "w")
my_file.puts "<! doctype html>
my_file.puts "<html><head>
my_file.puts "&met;meta name=\" viewport \ "content=\" initial-scale=1.0, user-scalable=no \ "/>"
my_file.puts "<style type=\" text/css \ ">"
my_file.puts "html {height:100%}"
my_file.puts "body {height:100%;margin:0px;padding:0px}"
my_file.puts "#map_canvas {height:100%}"
my_file.puts "</style>"
my_file.puts "&script type=\" text/javascript \ ""
my_file.puts "src=\" http://maps.google.com/maps/api/js?sensor=false \ ">"
my_file.puts "</script>"
my_file.puts "<script type=\" text/javascript \ ">"
my_file.puts "function initialize () {"
my_file.puts "var latlng=new google.maps.latlng (" + lat.to_s + "," + long.to_s + ");"
my_file.puts "var myoptions={"
my_file.puts "zoom:12,"
my_file.puts "center:latlng,"
my_file.puts "maptypeid:google.maps.maptypeid.hybrid"
my_file.puts "};"
my_file.puts "var map=new google.maps.map (document.getelementbyid (\" map_canvas \ "),"
my_file.puts "myoptions);"
my_file.puts "}"
my_file.puts "</script>"
my_file.puts "</head>"
my_file.puts "<body onload=\" initialize () \ ">"
my_file.puts "<div id=\" map_canvas \ "style=\" width:100%;height:100%\ "></div>"
my_file.puts "</body>"
my_file.puts "</html>"
puts "no geolocation data available."

The script in Listing 12 is simple to execute:

$./where-am-i.rb mtimjones

The resulting html file can be rendered by a browser,like this:

$firefox test.html

If no location information is available,This script will fail to execute;But if the implementation is successful,An html file will be generated, which can be read by the browser to render the map.Figure 5 shows the resulting map image.It shows parts of the front range in northern Colorado, USA.

Figure 5. Example of the image rendered by the script in Listing 12

With geography,You can also search twitter to identify twitter users and tweets associated with a particular location. The twitter search api allows the use of geocoded information to limit search results.The example in Listing 13 below extracts the user's longitude and latitude,Then use this data to get all tweets within 5 kilometers of the location.

Listing 13. Using longitude and latitude data to search for local tweets (tweets-local.rb)

#!/usr/bin/env ruby
require "rubygems"
require "twitter"
twitter.configure do | config |
screen_name=string.new argv [0]
a_user=twitter.user (screen_name)
if a_user.geo_enabled == true
long=a_user.status.place.bounding_box.coordinates [0] [0] [0]
lat=a_user.status.place.bounding_box.coordinates [0] [0] [1]
array tweets=twitter ::search.new.geocode (lat, long, "5mi"). fetch
tweets.each do | t |
puts t.from_user + "|" + t.text

The execution result of the script in Listing 13 is shown in Listing 14, which is a subset of the tweets posted by twitter users in the specified location range.

Listing 14. Querying tweets for this local area within 5 kilometers of my location.

$./tweets-local.rb mtimjones
breesesummer | @daltonols did he answer u
longmontradmon | 60 cpm, 0.4872 usv/h, 0.6368 usv/h, 2 time (s) over natural radiation
graelston | on every street there is a memory;a time and place we can never be again.
breesesummer | #i "minafight with @daltonols to see who will marry @thecodysimpson i will
marry him !!!:/
_jenniejune_ | ok i "m done, goodnight everyone!
breesesummer | @daltonols same
_jenniejune_ | @sylquejr sleep well!
breesesummer | @daltonols ok let "s see what he says
longmontradmon | 90 cpm, 0.7308 usv/h, 0.7864 usv/h, 2 time (s) over natural radiation
breesesummer | @thecodysimpson would u marry me or @daltonols
natcapsolutions | rt hlovins:the scientific rebuttal to the silly forbes release this
morning:misdiagnosis of surface temperatu ... http://bit.ly/nrpljl

Keep going

  • Previous A summary of javascript learning every day (basic knowledge)
  • Next Simple learning for statement loop structure in JavaScript