Focus Changes

Altering course:

    In the beginning of this blog, I stated that I was going to do this hard mode.  NPC stations in high sec and no real outside help.  Well that was a fun exercise. Next?

    Just a note:  I have been up for greater than 24 hours at this point, so the stream of thought is tending to sputter more than it should.

POSsibilities:

    I went ahead and dropped a POS in a quiet 0.5 system last week.  I got it all setup and moved most of my production there.  It's more expensive to have Red Frog ship out to me now, and I still have to cover an additional 2 jumps to get it into my POS.  I need to train a character into an Orca to make these last couple of jumps in as few trips as possible.   I still have a couple of copy jobs etc. hanging around my old home.

   So now I have all this possibility.  I am training up a component builder/BPO researcher for my own library and to support my invention toons.  This takes advantage of the ME/PE research slots I get for free, and allows me to increase profit margins without reducing ISK/hr of my inventors.

    I have thought about a lot.  I thought about buying all the parts that I would need to start building Orcas.  But with an initial buy-in of 10-12 billion and profit under 10% for a very well researched BPO, it just didn't seem worth while to me.

    All of these possibilities have left me needing better tools.  I find myself spending more time writing tools than playing most nights anymore.

    I also need to find more time to spend in Faction Warfare with my main.  Until I have a steady income with minimal work, I am not seeing a lot of time for it though.

Tool Development:

Overview:

    While the main focus of this blog is not on my tools, they are part of my decision making process.  I have chosen to write these tools in Java for several reasons.

• Cross-platform:
• Java is "natively" a cross-platform language.
• I needed/wanted to learn it for work:
• I work as a sometimes programmer, and was looking to write a cross-platform data extraction/analysis tool.
• I wanted a language with better libraries than Perl or C/C++.
• Perl libraries tend to be a little more unpolished and hap hazard.
• C/C++ support for things like XML and JSON, in the freeware department, Are lacking or non existent.
• Yes I will take flack for this, but after working on a project and compiling all the libraries using mingw through a cygwin terminal (for unix like commands needed by the make files) and realizing that cygwin adds a dll dependancy to your executables.
• Free tools, like Window Builder.

    I started with a simple tool to do invention calculations and spit out shopping lists, has turned into an interlocked toolkit, that just keeps growing.

Rapid Assembly:

    This was my initial tool idea, but a lot more complex.  Lots of options here, but when you boil them all down, the only tool currently with a GUI is my shopping list editor. Lots of options to dial in manufacturing and invention costs.  Has some functions to make historical and other graphs.

   One of the issues using this tool is that the database holding its blueprint information only holds the current SDE release, so you can not automatically see the cost of change of materials.

Profiteer:

    Command line tool to generate statistical information for T1 manufacturing over a period of time.  Output is in the form of csv's with product name/date.

    Groups blueprints by general market groups.  Can be used to filter down the list to a single market group.  Depth of the market group into tree does not matter (i.e. both Ammo & Charges and Cruise Missiles are valid.)

Invention Profiteer:

    Much like Profiteer, but adds in all the variations of the invention process.  It cycles through all available decryptors and meta items for each T2 blueprint in the selected market group.  I need to find some better ways of estimating true velocity.  I tried entering the Large Remote Energy Transfer Array II market, and am still holding onto ~300-500 M stock in sales orders.  On paper, this product looked very good, but the velocity is turning out to be very low.

killPuller:

    This is an attempt to pull kill data from zkillboard.com.  All of the kill data, along with some related price information is stuffed into a giant database.  From the database I plan to get item destroyed data to help with making market decisions.

killAnalyzer:

    This is a tool that I have not yet started working on.  I will need to finish killPuller enough to reliably extracts the large dataset for me before I can start working on this.  This will be the second GUI tool.

    Below is a list of desired functionality:

• ALOD finder.
• Battle finder / reporter.
• Should be able to search the kill data and find large battles.
• Should be able to create different battle reports.
• A per side isk lost over time graph, to give an idea how the battle progressed.
• Summary statistics
• Damage done
• Damage taken
• Kills
• Losses
• Item statistics
• Isk lost (destroyed)
• Isk salvageable (dropped)
• Galaxy Map graphs of various statistics
• Items destroyed
• Kills
• Pod Kills
• Gate camps

Industrial Accountant:

    I need to design a smart application to read in API data to find actual buy/sell prices on a unit by unit basis.

• Trace the material from purchase through industry jobs and finally into the sell orders.
• Categorize other purchases, both automagically and manually.
• Ability to graph data in various was

Conclusion:

    I need to get busy :).  Faction warfare, industrial warfare, and programming.  Lots to do, lets see if I can find the time to do much of this.

ZKB Json parsing in Java

Json parsing in Java using gson:

   While looking into pulling kill data using zkillboard's API, I had some trouble learning how to parse Json.  This is partially because its the first time that I was using Json, and partially because I am still new to Java as a whole.  Below is an example of how to parse kill data from ZKB.

    Some notes on parsing data from ZKB:

• You need ~15 second program pause after every call to ZKB.  This is not reflected in this code as it parses a single page of the call.
• Please see the ZKB Api page for more information on how to make the calls you want.

package com.JEves.test;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.lang.reflect.Type;
import java.net.InetSocketAddress;
import java.net.Proxy;
import java.net.ProxySelector;
import java.net.SocketAddress;
import java.net.URI;
import java.net.URL;
import java.net.URLConnection;
import java.util.List;

import com.btr.proxy.search.ProxySearch;
import com.btr.proxy.search.ProxySearch.Strategy;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.JsonDeserializationContext;
import com.google.gson.JsonDeserializer;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import com.google.gson.JsonParseException;

public class SimpleKbParser   implements JsonDeserializer {

 public String m_value;
 /**
  * @param args
  */
 @SuppressWarnings("deprecation")
 public static void main(String[] args) {
  ProxySearch proxySearch = new ProxySearch();
  proxySearch.addStrategy(Strategy.OS_DEFAULT); 
  proxySearch.addStrategy(Strategy.JAVA); 
  proxySearch.addStrategy(Strategy.BROWSER); 
  ProxySelector proxySelector = proxySearch.getProxySelector(); 

  if(proxySelector != null) {
   ProxySelector.setDefault(proxySelector); 
   URI home = URI.create("http://www.google.com"); 
   //System.out.println("ProxySelector: " + proxySelector); 
   //System.out.println("URI: " + home); 
   List proxyList = proxySelector.select(home); 
   if (proxyList != null && !proxyList.isEmpty()) { 
    for (Proxy proxy : proxyList) { 
     //System.out.println(proxy); 
     SocketAddress address = proxy.address(); 
     if (address instanceof InetSocketAddress) { 
      String host = ((InetSocketAddress) address).getHostName(); 
      String port = Integer.toString(((InetSocketAddress) address).getPort()); 
      System.setProperty("http.proxyHost", host); 
      System.setProperty("http.proxyPort", port); 
     } 
    } 
   }
  }
  URLConnection.setDefaultRequestProperty("Accept-Encoding", "gzip, deflate");
  URLConnection.setDefaultRequestProperty("User-Agent", "Valkrr Dragonsworn");
  BufferedReader reader;
  String itemUrl = "http://zkillboard.com/api/kills/regionID/10000002/";
  try {
   GsonBuilder gsonBuilder = new GsonBuilder();
      gsonBuilder.registerTypeAdapter(SimpleKbParser.class, new SimpleKbParser());
      Gson gson = gsonBuilder.create();
   reader = new BufferedReader(new InputStreamReader(new URL(itemUrl).openStream()));
   SimpleKbParser[] kills = gson.fromJson(reader, SimpleKbParser[].class);
   for(int i = 0; i < kills.length; i++) {
    System.out.println("Kill: " + i);
    System.out.println(kills[i].m_value + "\n");
   }
  }
  catch(Exception e) {
   e.printStackTrace();
  }
 }

 @Override
 public SimpleKbParser deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
  SimpleKbParser ret = new SimpleKbParser();
  int killID = ((JsonObject) json).get("killID").getAsInt();
  int solarSystemID = ((JsonObject) json).get("solarSystemID").getAsInt();
  String killTime = ((JsonObject) json).get("killTime").getAsString();
  int moonID = ((JsonObject) json).get("moonID").getAsInt();
  ret.m_value += "killID: " + killID;
  ret.m_value += "\nsolarSystemID: " + solarSystemID;
  ret.m_value += "\nkillTime: " + killTime;
  ret.m_value += "\nmoonID: " + moonID;
  return ret;
 }

}

I will go through the above class section by section.  I am assuming you will be copying this into an editor so I will be referring to line numbers.

Lines 1-23:

   Your basic setup and import steps.

Line 25:

   public class SimpleKbParser   implements JsonDeserializer<SimpleKbParser> {

  JsonDeserializer is what makes the magic easy an quick.  This basically allows you to have your storage class get 1 element at a time from the series of elements and have to worry about parsing just that one element.  

Lines 33-57:

   I do part of my programming behind a firewall and the other part outside of a firewall.  This piece of code allows the script to auto detect the correct proxy type and use that by default. This requires proxy-vole to be linked to the project, so if you are not worried about firewalls, remove these lines along with lines 15 and 16.

Lines 58-75:

   An extremely basic program for grabbing and printing kill data.

   The key line here is line 67:

      SimpleKbParser[] kills = gson.fromJson(reader, SimpleKbParser[].class);

    This line tells gson that you want to parse the BufferedReader with your json in it into an array of SimpleKbParser objects.

Lines 79-90:

   Here is the function overloaded from JsonDeserializer.  For this example I just pulled some of the simple information out of the json and added it to a member string to be later printed.  Depending on what you are trying to do with this information, you will have to decide what information you want.

int killID = ((JsonObject) json).get("killID").getAsInt();

   You have to cast the JsonElement json to JsonObject, then run the get(Element Tag) which returns a JsonElement.  You run the getAs<Type>() function on the resulting JsonElement to get the value in a usable format.

JsonArray attackers = ((JsonObject) json).get("attackers").getAsJsonArray();

for(int i = 0; i < attackers.size(); i++) {

   JsonObject attacker = attackers.get(i).getAsJsonObject();

}

   For array data inside a json element, you cast as above, but you get it as a JsonArray type.  You can then iterate over that array, getting each object as a JsonObject to make getting values easier.

Summary:

   So, we learned how to get ZKB data into our programs in just a few dozen lines of actual code, less that 100 total lines.  There are some things not covered here to keep it simple, line how to effectively iterate over pages in a query and pretty much any error checking.  All of the coding is done in Eclipse build v21.1.0 using the ADT install package.  Other compilers might have compiler specific requirements that could prevent the above code from compiling.

   My source code can be found at Rapid Assembly.  This include git source repository and an occasional executable of my tool.  I have not made much use of the zkillboard parser yet, but I have plans to do so as part of a product selection GUI.