Latency and Throughput in Center versus Edge Stream Processing

Earlier this year I finished my Computer Science Master’s thesis project at Chalmers University of Technology in Gothenburg, Sweden, which has the title “Latency and Throughput in Center versus Edge Stream Processing: A Case Study in the Transportation Domain”. The project report as well as most of the code are available on my Gitlab repository msc-thesis-streamprocessing.

Abstract:

The emerging Internet of Things (IoT) enables novel solutions. In this thesis report, we turn our attention to the problem of providing targeted accident notifications in near real-time. We use traffic data generated by Linear Road, a popular benchmark for stream processing engines, and simulate a possible real-world scenario in which connected cars continuously send position updates. We analyze this stream of position updates with the goal of identifying accidents, so that targeted accident notifications can be issued. This means that only cars within a certain distance of a known accident site will be notified.

In a real-world scenario, the required data analysis could be performed in different ways. We consider two possibilities. First, position reports are aggregated by road side units (RSUs) and forwarded to a central server. Afterwards, the results are sent back to the cars, again involving RSUs for transmission. We refer to this as center stream processing. Second, all data analysis is performed on RSUs. An RSU is less powerful than a server. However, RSUs are located much closer to the cars than a central server. We refer to this case as edge stream processing. Performing computations directly on RSUs has the benefit that the cost of the roundtrip time for data transmission from RSUs to the server and back will be avoided. We use a contemporary stream processing engine for data analysis, and compare latency and throughput of an implementation of our solution to the accident notification problem in both cases.

PLC Factory – Automating Large-Scale PLC Development

I spent this summer working at the European Spallation Source (ESS) in Lund, Sweden. My contribution was the creation of PLC Factory, a tool that automates development for programmable logic controllers (PLCs). A paper on this project is forthcoming.

The code of PLC Factory is available on the Bitbucket account of ESS. PLC Factory was developed as FOSS. Thus, I can make the code I wrote available on my private GitLab account as well.

The draft of the PLC Factory paper is likewise available on GitLab. Here is the full title information including the abstract:

PLC FACTORY: AUTOMATING ROUTINE TASKS IN LARGE-SCALE PLC SOFTWARE DEVELOPMENT

Authors:
G. Ulm, D. Brodrick, N. Levchenko, F. Bellorini

Abstract:
At the European Spallation Source in Lund, Sweden, the entire facility including all its instruments will be controlled by a large number of programmable logic controllers (PLCs). Programming PLCs, however, entails a significant amount of repetition. It is thus an error-prone and time-consuming task. Given that PLCs interface with hardware, this involves economic aspects as well, due to the fact that programming errors may cause damage to equipment. With PLC Factory, we managed to automate repetitive tasks associated with PLC programming and interfacing PLCs from EPICS. This tool is being adopted at ESS, and has shown potential for a large increase in productivity compared to the status quo. We describe PLC Factory as well as its embedded domain-specific programming language PLCF#, which it is built upon.

CodingBat: Java. Map-2

Here are my solutions to the Map-2 section on CodingBat.

word0:

public Map<String, Integer> word0(String[] strings) {
  
  Map<String, Integer> map = new HashMap<String, Integer>();
  
  for (int i = 0; i < strings.length; i++) {
    map.put(strings[i], 0);
  }
  
  return map;
}

wordLen:

public Map<String, Integer> wordLen(String[] strings) {
    
  Map<String, Integer> map = new HashMap<String, Integer>();
  
  for (int i = 0; i < strings.length; i++) {
    String tmp = strings[i];
    map.put(tmp, tmp.length());
  }
  
  return map;
}

pairs:

public Map<String, String> pairs(String[] strings) {
  
  Map<String, String> map = new HashMap<String, String>();
  
  for (int i = 0; i < strings.length; i++) {
    String tmp   = strings[i];
    String first = String.valueOf(tmp.charAt(0));
    String last  = String.valueOf(tmp.charAt(tmp.length() - 1));
    map.put(first, last);
  }
  
  return map;
}

wordCount:

public Map<String, Integer> wordCount(String[] strings) {
  
  Map<String, Integer> map = new HashMap<String, Integer>();
  
  for (int i = 0; i < strings.length; i++) {
    
    String tmp = strings[i];
    
    if (map.containsKey(tmp)) {
      int count = map.get(tmp);
      map.put(tmp, count + 1);
    } else {
      map.put(tmp, 1);
    }
    
  }
  return map;
}

firstChar:

public Map<String, String> firstChar(String[] strings) {
  
  Map<String, String> map = new HashMap<String, String>();
  
  for (int i = 0; i < strings.length; i++) {
    
    String key = String.valueOf(strings[i].charAt(0));
    
    if (map.containsKey(key)) {
      String val = map.get(key) + strings[i];
      map.put(key, val);
    } else {
      map.put(key, strings[i]);
    }
    
  }
  return map;
}

wordAppend:

public String wordAppend(String[] strings) {
 
 Map<String, Integer> map    = new HashMap<String, Integer>();
 String               result = "";
 
 for (int i = 0; i < strings.length; i++) {
   
   String key = strings[i];
   
   if (map.containsKey(key)) {
     int val = map.get(key);
     val++;
     if (val % 2 == 0) {
      result += key;
     }
     map.put(key, val);
   } else {
     map.put(key, 1);
   }
   
 }
 
 return result;
}

wordMultiple:

public Map<String, Boolean> wordMultiple(String[] strings) {
  
  Map<String, Integer> counts = new HashMap<String, Integer>();
  Map<String, Boolean> result = new HashMap<String, Boolean>();
  
  for (int i = 0; i < strings.length; i++) {
    String key = strings[i];
    
    if (counts.containsKey(key)) {
      int val = counts.get(key);
      val++;
      counts.put(key, val);
    } else {
      counts.put(key, 1);
    }
    
    result.put(key, counts.get(key) >= 2);
  }
  
  return result;
}

allSwap:

public String[] allSwap(String[] strings) {
  
  Map<String, Integer> map = new HashMap<String, Integer>();
    
  for (int i = 0; i < strings.length; i++) {
    
    String key = String.valueOf(strings[i].charAt(0));
    
    if (map.containsKey(key)) {
      
      // swap
      int    pos   = map.get(key); 
      String tmp   = strings[pos];
      strings[pos] = strings[i];
      strings[i]   = tmp ;
      
      // delete
      map.remove(key);
      
    } else {
      map.put(key, i);
    }
    
  }
  
  return strings;
}

firstSwap:

public String[] firstSwap(String[] strings) {
  
  Map<String, Integer> map = new HashMap<String, Integer>();
    
  for (int i = 0; i < strings.length; i++) {
    
    String key = String.valueOf(strings[i].charAt(0));
    
    if (map.containsKey(key)) {
      
      int val = map.get(key);
      if (val == -1) {
        continue;
      }
      
      // swap
      int    pos   = map.get(key); 
      String tmp   = strings[pos];
      strings[pos] = strings[i];
      strings[i]   = tmp ;
      
      // set a flag
      map.put(key, -1);
      
    } else {
      map.put(key, i);
    }
    
  }
  
  return strings;
}

CodingBat: Java. Map-1

Nick Parlante updated CodingBat with two new sets of Java exercises, which both focus on maps. My solutions for Map-1 are below.

mapBully:

public Map<String, String> mapBully(Map<String, String> map) {
  
  if (map.containsKey("a")) {
    String tmp = map.get("a");
    map.put("a", "");
    map.put("b", tmp);
  }
  
  return map;
}

mapShare:

public Map<String, String> mapShare(Map<String, String> map) {
  
  if (map.containsKey("a")) {
    String tmp = map.get("a");
    map.put("b", tmp);
  }
  
  map.remove("c");
      
  return map;
}

mapAB:

public Map<String, String> mapAB(Map<String, String> map) {
  
  if (map.containsKey("a") && map.containsKey("b")) {
    String tmp = map.get("a") + map.get("b");
    map.put("ab", tmp);
  }
  
  return map;
}

topping1:

public Map<String, String> topping1(Map<String, String> map) {
  
  if (map.containsKey("ice cream")) {
    map.put("ice cream", "cherry");
  }
  
  map.put("bread", "butter");
  
  return map;
}

topping2:

public Map<String, String> topping2(Map<String, String> map) {
  
  if (map.containsKey("ice cream")) {
    map.put("yogurt", map.get("ice cream"));
  }
  
  if (map.containsKey("spinach")) {
    map.put("spinach", "nuts");
  }

  return map;
}

topping3:

public Map<String, String> topping3(Map<String, String> map) {
  
  if (map.containsKey("potato")) {
    map.put("fries", map.get("potato"));
  }
  
  if (map.containsKey("salad")) {
    map.put("spinach", map.get("salad"));
  }

  return map;
  
}