Facebook Group Archiver

This app helps us archive all posts from a facebook group along with its metadata and generate analytics to improve user involvement.

Scripts

fb_logger.py

• Archives posts, comments, likes from the group
• Parses out links from posts and sends it to kippt

db/dbcreate.sql

• Create table statements for tables - posts, comments, likes, links, user, user_points
• Functions to insert into / update tables - posts, comments, likes, links
• Functions to create users and calculate points for each user - user, user_points

cronjob.py

• Should run every 'few' minutes to update the database
• Basically calls fb_logger.py along with the facebook graph api url

batchscript.py

• Fetches all of the posts in the group since its beginning - for first time update.

insert_users.py

• This script is run for the first time to update User db with group members
• The Facebook API only returns ~5000 members, so usercron.py ensures daily updates of the User table

usercron.py

• This cron is run daily to update the User table
• Fetches top 50 users(recently added) from the facebook group member list and adds them to the user table
• To cope for the old missed out users because of ~5000 limit of the API it finds users in user_points not in user table, and populates their details

Configuration

config/default_properties.cfg

• facebook group links - graph api url and members list fql url
• location of info log and error log files
• mysql database configuration
• kippt configuration
• cronjob interval (in minutes)

Commands

cronjob

python cronjob.py config/<properties_filename>.cfg

usercron

python usercron.py config/<properties_filename>.cfg

insert_users

python insert_users.py config/<properties_filename>.cfg

batchscript

python batchscript.py config/<properties_filename>.cfg [facebook group graph api url(optional)]