Social and Information Network Analysis
Pointers to data and code
Stanford Large Network Dataset Collection
Coauthorship and Citation Networks
- AS Graphs:
AS-level connectivities inferred from Oregon route-views, Looking glass data and Routing registry data
- Yelp Review Data:
reviews of the 250 closest businesses for 30 universities for students and academics to explore and research
Prosper peer to peer money lending dataset
- Money Lending
Data: Lenders ask for loans and people bid (price, interest rate) on
loans to fund.
- Youtube data:
YouTube videos as nodes. Edge a->b means video b is in the related video list (first 20 only) of a video a.
Amazon product copurchasing networks and metadata
Data: The data was collected by crawling Amazon website and contains
product metadata and review information about 548,552 different products
(Books, music CDs, DVDs and VHS video tapes).
page to page link data: A list of all page-to-page links in Wikipedia
- DBPedia: The
DBpedia data set uses a large multi-domain ontology which has been derived from Wikipedia.
- Edits and
talks: Complete edit history (all revisions, all pages) of Wikipedia since its inception till January 2008.
Who trusts whom data at Trustlet
Mark Newman's pointers
Munmun De Choudhury's pointers
data: Flickr Image Dataset, YouTube Dataset, Digg Dataset
(Social Media), Engadget Dataset (online communities), Del.icio.us
Dataset (Social bookmarking)
Reality Commons data
data: Several mobile data sets that contain the dynamics of several communities of about 100 people each.
Stanford Foursquare Place Graph Dataset
- Every day millions of people check-in to the places they go on Foursquare and in the process create vast amounts of data about how places are connected to each other. We call this set of interconnections the Place Graph, and provide a sample of this data for 5 major US cities. This dataset contains metadata about 160k popular public venues, and 21m anonymous check-in transitions (or trips between venues). You'll have to sign an agreement to gain access; contact Jure for more information.
- GitHub is one of the most popular platforms for sharing code online. There are two sources for GitHub data:
GHTorrent maintains a relational model of GitHub activity data and offers archives for download. Both MySql and MongoDB dumps are available.
- GitHub Archive
GitHub archive is the comprehensive source of all GitHub events starting from February 12, 2011. It is officially provided by GitHub and includes 18 event types, which range from new commits and fork events, to opening new tickets, commenting, and adding members to a project. The activity is aggregated in hourly archives, which you can access with any HTTP client.
Google Local Dataset
The dataset contains ratings and reviews of local businesses obtained from Google, courtesy Julian McAuley. We ask that you credit Julian if you choose to publish your work using this dataset.
- Bitcoin is a digital currency invented in 2008 and operates on a peer-to-peer system for transaction validation. This decentralized currency is an attempt to mimic physical currencies in that there is limited supply of Bitcoins in the world, each Bitcoin must be "mined", and each transaction can be verified for authenticity. Bitcoins are used to exchange every day goods and services, but it also has known ties to black markets, illicit drugs, and illegal gambling transactions. The dataset is also very inclined towards anonymization of behavior, though true anonymization is rarely achieved.
- The Bitcoin dataset captures transaction-level information. For each transaction, there can be multiple senders and multiple receivers as detailed here. This dataset provides a challenge in that multiple addresses are usually associated with a single entity or person. However, some initial work has been done to associated keys with a single user by looking at transactions that are associated with each other (for example, if a transaction has multiple public keys as input on a single transaction, then a single user owns both private keys). The dataset provided provides these known associations by grouping these addresses together under a single UserId (which then maps to a set of all associated addresses).
- Key Challenge Questions:
- Can we detect bulk Bitcoin thefts by hackers? Can we track where the money went after thefts?
- Can we detect illicit transactions based on Bitcoin transaction behavior? What sort of graph patterns emerge?
- Can we detect attempts at money laundering (called a "mixing service" in Bitcoin)
- Can we detect money laundering attempts and the people who use them? Note: Current Bitcoin mixing services tend to mix Bitcoins amongst all the people who bother to use a mixing service so does the mixing service actually obfuscate anything?
- Can we trace back the originator of these laundering attempts?
- Can we detect currency manipulation (hackers try to destabilize Bitcoin currency exchanges to deflate prices)
- Is Bitcoin gaining traction or losing traction among the regular population for use as a regular digital currency?
- It is Bitcoin best practice to generate and use a new address with every transaction. Is this practice followed? If not, then what can we learn from this?
- Can we identify and extract organizational behavior amidst the Bitcoin transactions?
- Can we determine which Bitcoin addresses belong to a single entity? While the initial pass over the data have yielded some resolution of entities, can we further improve this mapping?
MOOC Forums Dataset
- All data from Stanford's courses on Coursera and NovoEd is available. For Coursera format details see this page. For an explanation of data available from Stanford courses offered on our OpenEdX platform, see Datastage. To request any of the data, fill in this form. For more details, please contact Jure.
- A number of (relatively) new OpenEdX data are now available on datastage.stanford.edu. These include both data that the OpenEdX platform collects, and tables that result from computations over that base data. In addition, processes are now in place to keep the data current on a daily to weekly basis (Coursera and NovoEd data is integrated at the end of each course)
- In summary, the additions are:
- ActivityGrade: Assignment grades Includes right/wrong for each problem part, the learners' solution choice for each answer, and the first and last solution submission times.
- Cumulative assignment performance per learner
- 'Raw' final grades, updated at the end of courses.
- Demographic information: country, gender,year_of_birth, and level_of_education. This information is not fully populated because its provision is optional
- A much slimmed view of the OpenEdX tracking log events. The view only includes fields that are currently in use by the platform.
- An anonymized record of the forum from each class.
- The country of origin of each class participant (by IP address).
C++ libary for working with massive network datsets (Windows, Linux, Mac)
Program for large network analysis (Windows or Linux via Wine)
Python package for the study of the structure of complex networks
Graph visualization software
Exploratory data analysis and visualization tool for graphs and networks
Software framework for information visualization (Linux, MacOSX, Windows)
Software for social network analysis (Windows)
Large-scale network analysis, modeling and visualization toolkit
Tools for fitting heavy-tailed distributions to data
Some websites that may be interesting to do analysis on: