Designing Modern Web-Scale Applications

ECE1724, Winter 2020
University of Toronto

Instructor: Ashvin Goel
Course Room: BA4164
Course Time: Fri, 1-3 pm
Start Date: Jan 10, 2020

HomeAccessing PapersPresentation FormatProject FormatProject Ideas

Project Ideas

Some suggested projects are described below. You are also welcome to pick a project of your own choice. However, it is important for you to think about the following questions regarding your project before starting any design and implementation: 1) what problem are you addressing, 2) what is interesting/novel about your approach, 3) what metrics and testing method will you use for evaluation, and 4) what results do you expect from the evaluation. Your project reports and final presentation will be evaluated based on these criteria.

All the projects described below are based on work being done by the instructor's group. Please talk to the instructor about more details regarding the projects.

Please make sure to get a confirmation about your project from the instructor before starting the project.

  1. Improving Availability in the Caracal Database

    Most web application store their data in databases today. Thus cloud-based database deployments often use various forms of data replication and failover mechanisms to improve availability and guard against catastrophic failures.

    The instructor's group has been working on a high-performance, in-memory database called Caracal that is designed to scale well for distributed transactions. Currently, this database has limited support for various reliability features such as logging, checkpointing, replication and failover. In this project, you will be working on one of the projects shown below for improving the reliability of Caracal. Caracal is implemented in C++, so you will need significant experience with C++ development. Talk to the instructor for more details.

    1. Logging and Checkpointing

      In this project, you will implement support for logging and checkpointing a single-node database. This implementation will allow recovering from database crashes and single node failures.

    2. Replication

      In this project, you will implement support for replicating a single-node database to a backup node, and then integrate the replication scheme with either Zookeeper or Raft. This implementation will allow issuing snapshot read queries to backup nodes.

    3. Failover

      In this project, you will implement support for failover. This implementation will allows handling master failure, so that a backup node will take over database operation. This project can be performed independently of the replication project described above.

  2. Replication in the Hailstorm Storage System

    Today, cloud-based systems often use log-structured merge tree (LSM) based storage systems. These storage systems provide both fast read and write accesses by maintaining data in two or more separate structures, each optimized for its storage media.

    The instructor's group has been working on a rack-level, LSM-based storage system called Hailstorm that is designed to balance CPU and storage load across the machines in the rack. Currently, the storage system does not support data replication. In this project, you will be working on implementing replication for Hailstorm, similar to the replication mechanisms used in GFS and Azure. Hailstorm is implemented in Java, and so you will need significant experience with Java development. Talk to the instructor for more details.

  3. Skew in Distributed Graph Mining Applications

    Graph mining algorithms help discover complex structural patterns, such as cliques and motifs in graphs, enabling applications such as analyzing communities in social networks.

    The instructor's group has been working on a distributed graph mining system called Tesseract that is designed for large evolving graphs. As graph updates arrive, Tesseract detects new graph patterns with low delay and streams them for further processing. Scaling distributed graph mining is challenging because graphs can be highly skewed (e.g., some vertices may have 1000s of neighbors) and updates can also be skewed (e.g., some vertices see most of the updates). In this project, you will be evaluating Tesseract's behavior with skewed workloads and implementing methods for mitigating skew. Tesseract is implemented in Java, and so you will need significant experience with Java development.