Research Projects on Google App Engine

By Andrea Held, Program Manager, Google University Relations

Cross-posted on the Google Developers Blog

Last spring Google University Relations announced an open call for proposals for Google App Engine Research Awards. We invited academic researchers to use Google App Engine for research experiments and analysis, encouraging them to take advantage of the platform’s ability to manage heavy data loads and run large-scale applications. Submissions included exciting proposals in various subject areas from mathematics, computer vision, bioinformatics, climate and computer science. We have selected seven projects that have the potential to impact people’s lives by making community seismic networks affordable, creating individualized DNA profiles, collecting useful local data through social media, and by understanding global climate trends, just to mention a few.

We have donated $60,000 in Google App Engine credits to each of these projects recognizing the innovation and vision of the Principal Investigator and his collaborators. Congratulations to all of them!

Below is a brief introduction of the award recipients and their research. We look forward to learning about their progress and will share the news right here. Stay tuned!

K. Mani Chandy, Simon Ramo Professor and Professor of Computer Science, California Institute of Technology
Project title: Cloud-based Event Detection for Sense and Response
Description and research goals: We developed an App Engine-based sense and response platform for the Community Seismic Network (CSN) project. CSN's goals include measuring seismic events with finer spatial resolution than previously possible and developing a low-cost alternative to traditional seismic networks, which have high capital costs for acquisition, deployment, and ongoing maintenance. We are working on generalizing our implementation and experience to provide a system for other members of the community to use in future sense and response applications.

Lawrence Chung, Associate Professor, The University of Texas at Dallas
Project title: Google App Engine: Software Benchmark and Simulation Forecaster
Description and research goals: An important consideration before migrating a company’s application software to Google App Engine is performance and operating cost.
Similarly, the Google App Engine organization would want to estimate Google App Engine’s resource usage and how well the particular resource allocation will meet the performance and cost requirements, as in the service level agreements (SLAs). This research project aims to develop a Google App Engine simulation forecaster - a tool for estimating the performance and cost of software operating on Google App Engine, and produce some important operational benchmark.

Julian Gough, Professor, University of Bristol, UK
Project title: Personalised DNA Analysis
Description and research goals: Personal genomics is still in its infancy and although it is easy, and relatively cheap to obtain personal genotype data, the available analysis is not personalised; it is the same for everybody. In this project we will set up a service powered by App Engine that provides personal DNA analysis specific to each individual. The proposed service does not focus on disease, but on identifying aspects of a healthy person that make them unique. What does your genome tell you about yourself that makes you special?

Ramesh Raskar, PhD, MIT Media Lab; Dr. Erick Baptista Passos, IFPI (Federal Institute of Technology, Brazil)
Project title: Vision Blocks
Description and research goals: Vision Blocks is a research project that aims to make computer vision available to everyone. Its primary goal is to develop tools for delivering computer vision to masses through an extensible visual programming language and an online application building and sharing system. We have a prototype HTML5 client that already performs computer vision tasks locally. Our goals for the next iterations include integration with App Engine for preprocessing of video streaming platforms.

Norman Sadeh, Professor, Director of Mobile Commerce Lab, School of
Computer Science, Carnegie Mellon University; Justin Cranshaw, PhD student, School of Computer Science, Hazim Almuhimedi, PhD student, School of Computer Science
Project title: Mapping the Dynamics of a City & Nudging Twitter Users
Description and research goals: We are working on two research
projects. The first is Livehoods in which we take a computational approach to analyzing large-scale trends in the ways people move through dense urban areas. Our goal is to find algorithmic ways of uncovering local collective knowledge about the city using social media. The second is “Nudging Twitter Users” in which we utilize quantitative and qualitative approaches to understand why people post things on Twitter they wish they had not, and also to understand the nature of these posts. Our objective is to develop tools that help nudge users to reduce the likelihood of those posts.

William Stein, Professor of Mathematics, University of Washington
Project title: Sage: Creating a Viable Free Open Source Alternative to Magma, Maple, Matlab, and Mathematica
Description and research goals: The goal is to create a highly scalable and resilient website through which very large numbers of people can use Sage. This is the next step.

Enrique Vivoni, Associate Professor, Hydrologic Science, Engineering & Sustainability, Arizona State University; Dr. Giuseppe Mascaro, Research Engineer; Jyothi Marupila, Graduate Student; Mario A. Rodriguez, Software Engineer
Project title: Cloud Computing-Based Visualization and Access of Global Climate Data Sets
Description and research goals: Our project uses Google App Engine for analyzing global climate data within the Google Maps API. At this stage, we are able to generate loads from the Global Land Data Assimilation Systems (GLDAS) climate model into the Google App Engine datastore. We select the climate variable to be used and aggregate data at different spatial resolutions. We are using Google App Engine Task Queue API to load large files. For the presentation layer, we are using Django templates to integrate the display of many data points in the Google Maps API. Our objective is to provide scientific data on global climate trends by allowing map-based queries and summaries at the appropriate resolutions. Sample Map

Currently, no further rounds for Google App Engine Research Awards have been planned. We will announce any updates to the program on our website.

GIT Merge conflict solving with kdiff3

First we will create a test repository and a test file on which we will create a merge conflict later on.
(in this example, the commands are executed with PowerShell in Windows, but the GIT commands are exactly the same on any other shell or OS)

• Creation of test repository
  

  > cd \tempmd mergetestcd mergetest
  > git init

• Creation of test file
  

  > 'Original line' | Out-File file.txt
  

• Commit into repository
  

  > git add file.txt
  > git commit -m "Original checkin"

• Current repository state: (using the tool GIT Extensions)
  

Now we will create a new branch A, and update the file.

• Create new branch
  

  > git branch branch-A
  > git checkout branch-A

• Update file
  

  > 'Branch A line' | Out-File file.txt

• Commit into repository
  

  > git add file.txt
  > git commit -m "Branch A change"

• Current repository state:
  

Now we will create a new branch B, and update the file with a conflicting change.

• Create new branch, starting from the original master branch
  

  > git checkout master
  > git branch branch-B
  > git checkout branch-B

• Update file
  

  > 'Branch B line' | Out-File file.txt

• Commit into repository
  

  > git add file.txt
  > git commit -m "Branch B change"

• Current repository state:
  

All this was a preparation for what we will do now: merge the conflicting change of branch-B into branch-A.

• Select the target branch (in this case branch A)
  

  > git checkout branch-A

• Merge the source branch (in this case branch B)
  

  > git merge branch-B

• This results in a merge conflict:
  

  Auto-merging file.txt
  CONFLICT (content): Merge conflict in file.txt
  Automatic merge failed; fix conflicts and then commit the result.

• This conflict can be resolved by starting your mergetool
  

  > git mergetool

• It gives you a message before it launches your mergetool (in my case kdiff3):
  

  Merging:
  file.txt
  
  Normal merge conflict for 'file.txt':
    {local}: modified file
    {remote}: modified file
  Hit return to start merge resolution tool (kdiff3):

• You get a window containing 4 versions of the same file:
• Top left: "Base".  This is the original version of the file, the last shared version between what became later branch-A and branch-B.
• Top Middle: "Local". This is the target branch version of the file.  It is called "Local", because this is the currently selected branch (using the last git checkout command).
• Top Right: "Remote". This is the source branch version of the file. It is called "Remote", as apposed to "Local".
• Bottom: "Output". This is the merged version of the file. After successful completion of the merge, this will become the new version of the target branch.  In this pane, you can edit the text, or you can right click on any merged part and select what version(s) of the original file you want to include in the output.  In this case, I decide to include the changes done in both branch-A and branch-B.
  
  
• Close the kdiff3 application (saving the file), and commit the merge:

  > git commit
  (accept the default commit message)

• This brings us to the following repository state:
  
• We can check if the content of the test-file is what we need:
  

  > Get-Content file.txt
  Branch A line
  Branch B line
  

   

FxGqlC: Added aggregation functions ENLIST and ENLISTDISTINCT

Added aggregation functions ENLIST and ENLISTDISTINCT.
ENLIST creates a string value containing the list of all string values in their original order.
ENLISTDISTINCT creates a similar list, but the dupplicate values are removed, and the list is ordered. 

SELECT [Winner], COUNT(*), ENLIST([Tournament])
   FROM ['SampleFiles/Tennis-ATP-2011.csv' -Heading=On]
   GROUP BY [Winner] ORDER BY 2 DESC
SELECT [Winner], COUNT(*), ENLISTDISTINCT([Tournament])
   FROM ['SampleFiles/Tennis-ATP-2011.csv' -Heading=On]
   GROUP BY [Winner] ORDER BY 2 DESC

This feature is added to FxGqlC in v2.5-alpha5.

FxGqlC: Added new function 'PREFIX'

Added (non-aggregation) text function PREFIX to return the common prefix of two strings.
An aggregation function PREFIX (with 1 argument) was already added in v2.4.

SELECT PREFIX('0032478123456', '0032478654321')  
-- returns '0032478'

This feature is added to FxGqlC in v2.5-alpha5.

Advanced Power Searching with Google -- Registration Opens Today

Posted by Daniel Russell, Über Tech Lead for Search Quality and User Happiness

Cross-posted at Inside Search Blog

What historic cafe inspired a poem by a Nobel Laureate? In the last three barista world championships, which winners did not use beans from their home country? If you were preparing a blog post on “Curious Trivia of Coffee Culture,” how would you find the answers to these questions? What else would you discover? Now you can sign up for our Advanced Power Searching with Google online course and find out.

Building on Power Searching with Google, Advanced Power Searching with Google helps you gain a deeper understanding of how to become a better researcher. You will solve complex search challenges similar to those I pose in my blog, or a Google a Day, and explore Google’s advanced search tools not covered in the first class.

Oftentimes the most intriguing questions invite you to explore beyond the initial answer, and there’s no single correct path to get there. When looking for questions that can’t be solved with a single query, “search” can quickly turn into “research.” Google Search offers a palette of tools to help you dive deeper into the web of knowledge.

Visit www.powersearchingwithgoogle.com to learn more about our online search courses, and review our search tips on the Power Searching with Google Quick Reference Guide. Advanced Power Searching begins on January 23 and ends on February 8th.

Conference Report: Workshop on Internet and Network Economics (WINE) 2012

Posted by Vahab Mirrokni, Research Scientist, Google Research New York

Google regularly participates in the WINE conference: Workshop on Internet & Network Economics. WINE’12 just happened last week in Liverpool, UK, where there is a strong economics and computation group. WINE provides a forum for researchers across various disciplines to examine interesting algorithmic and economic problems of mutual interest that have emerged from the Internet over the past decade. For Google, the exchange of ideas at this selective workshop has resulted in innovation and improvements in algorithms and economic auctions, such as our display ad allocation.

Googlers co-authored three papers this year; here’s a synopsis of each, as well as some highlights from invited talks at the conference:

Budget Optimization for Online Campaigns with Positive Carryover Effects
This paper first argues that ad impressions may have some long-term impact on user behaviour, and refers to an older WWW ’10 paper. Based on this motivation, the paper presents a scalable budget optimization algorithm for online advertising campaigns in the presence of Markov user behavior. In such settings, showing an ad to a user may change their actions in the future through a Markov model, and the probability of conversion for the ad does not only depend on the last ad shown, but also on earlier user activities. The main purpose of the paper is to give a simpler algorithm to solve a constrained Markov Decision Process, and confirms this easier solution via simulations on some advertising data sets. The paper was written when Nikolay Archak, a PhD student at NYU business school, was an intern with the New York market algorithms research team.

On Fixed-Price Marketing for Goods with Positive Network Externalities
This paper presents an approximation algorithm for marketing “networked goods” and services that exhibit positive network externalities - for example, is the buyer's value for the goods or service influenced positively by other buyers owning the goods or using the service? Such positive network externalities arise in many products like operating systems or smartphone services. While most of previous research is concerned with influence maximization, this paper attempts to identify a revenue maximizing marketing strategy for such networked goods, as follows: The seller selects a set (S) of buyers and gives them the goods for free, then sets a fixed per-unit price (p), at which other consumers can buy the item. The strategy is consistent with practice and is easy to implement. The authors use ideas from non-negative submodular maximization to find the optimal revenue maximizing fixed-price marketing strategy.

The AND-OR game: Equilibrium Characterization
Yishay Mansour, former Visiting Faculty in Google New York, presented the results; he first argued that the existence and uniqueness of market equilibria is only known for markets with divisible goods and concave or convex utilities. Then he described a simple market AND-OR game for divisible goods. To my surprise, he showed a class of mixed strategies are basically the unique set of randomized equilibria for this market (up to minor changes in the outcome). At the end, Yishay challenged the audience to give such characterization for more general markets with indivisible goods.

Kamal Jain of Ebay Research gave an interesting talk about mechanism design problems, inspired by application in companies like Ebay and Google. In one part, Kamal proposed "coopetitive ad auctions" for settings in which the auctioneer runs an auction among buyers who may cooperate with some advertisers, and at the same time compete with others for sealing advertising slots. He gave context around "product ads"; for example, a retailer like Best Buy may cooperate with a manufacturer like HP to put out a product ad for an HP computer sold at Best Buy. Kamal argued that if the cooperation is not an explicit part of the auction, an advertiser may implicitly end up competing with itself, thus decreasing the social welfare. By making the cooperation an explicit part of the auction, he was able to design a mechanism with better social welfare and revenue properties, compared to both first-price and second-price auctions. Kamal also discussed optimal mechanisms for intermediaries, and “surplus auctions” to avoid cyclic bidding behavior resulted from running naive variants of first-price auctions in repeated settings.

David Parkes of Harvard University discussed techniques to combine mechanism design with machine learning or heuristic search algorithms. At one point David discussed how to implement a branch-and-bound search algorithm in a way that results in a "monotone" allocation rule, so that if we implement a VCG-type allocation and pricing rule based on this allocation algorithm, the resulting mechanism becomes truthful. David also presented ways to compute a set of prices for any allocation, respecting incentive compatibility constraints as much as possible. Both of these topics appeared in ACM EC 2012 papers that he had co-authored.

At the business meeting, there was a proposal to change the title of the conference from “workshop” to “conference” or “symposium” to reflect its fully peer-reviewed and archival nature, keeping the same acronym of WINE. (Changing the title to “Symposium on the Web, Internet, and Network Economics” was rejected: SWINE!) WINE 2013 will be held at Harvard University in Boston, MA, and we look forward to reconnecting with fellow researchers in the field and continuing to nurture new developments and research topics.

Using online courses in Spain to teach entrepreneurship

Posted by Francisco Ruiz Anton, Policy Manager, Google Spain

Cross-posted with the Policy by the Numbers Blog

At the end of the third quarter in 2012, roughly 25% of adults in Spain were out of work. More than half of adults under 24 years old are unemployed. Recent graduates and young adults preparing to enter the workforce face the toughest job market in decades.

The Internet presents an opportunity for growth and economic development. According to recent research, more than 100,000 jobs in Spain originate from the Internet and it directly contributes to the GDP with 26.7 billion euros (2.5%). That impact that could triple by 2015 under the right conditions.

One of those conditions is making high-quality education accessible, echoed by a recent OECD report on the youth labor market in Spain. This is no easy task. University degrees are in high demand, straining the reach of our existing institutions.

The web has become a way for learners to develop new skills when traditional institutions aren’t an option. Recent courses on platforms like Udacity, Coursera and edX have seen hundreds of thousands of students enroll and participate in courses taught by prestigious professors and lecturers.

Google is partnering with numerous organizations and universities in Spain to organize UniMOOC, an online course intended to educate citizens in Spain and the rest of the Spanish-speaking world about entrepreneurship. It was built with Course Builder, Google’s new open source toolkit for constructing online courses.

To date nearly 10,000 students have registered for the course, over two-thirds of them from Spain and one-third from 93 countries. It recently won an award for the “Most innovative project” in 2012 from the newspaper El Mundo.

Spain’s situation is not entirely unique in Europe. Policymakers across the continent are asking themselves how best to create economic opportunity for their citizens, and how to ensure that their best and brightest students are on a path toward financial success. Our hope is that the people taking this course will be more empowered with the right skills and tools to start their own businesses that can create jobs. They will push not only Spain, but Europe and the rest of the world towards economic recovery and growth.

The course is still running, and you’re able to join today.