QR code

Software Quality Award, 2016

  • comments

award

badge

This is the second year of the Software Quality Award. The prize is still the same—$4,096. The rules were changed a bit. Read on. BTW, 2015 is here.

Rules:

  • One person can submit only one project.

  • Submissions are accepted until the September 1, 2016.

  • I will check the commit history to make sure you’re the main contributor to the project.

  • I reserve the right to reject any submission without explanation.

  • All submissions will be published on this page (including rejected ones).

  • Results will be announced October 15, 2016 on this page and by email.

  • The best project will receive $4,096.

  • Final decisions will be made by me and are not negotiable (although I may invite other people to help me make the right decision).

  • Winners that received any cash prizes in previous years can’t submit again.

Each project must be:

  • Open source (in GitHub).

  • At least 10,000 lines of code.

  • At least one year old.

  • Object-oriented (that’s the only thing I understand).

The best project is selected using this criteria.

What doesn’t matter:

  • Popularity. Even if nobody is using your product, it is still eligible for this award. I don’t care about popularity; quality is the key.

  • Programming language. I believe that any language, used correctly, can be applied to design a high-quality product.

  • Buzz and trends. Even if your project is yet another parser of command line arguments, it’s still eligible for the award. I don’t care about your marketing position; quality is all.

By the way, if you want to sponsor this award and increase the bonus, email me.

60 projects submitted so far (in random order):

15 Oct 2016: I asked one of our Java developers to do a preliminary analysis of all projects. This is the report he sent me back today: award-2016.txt (you can find your project there). I added my comments to his pluses and minuses (see them right in that text file). Based on his opinion and a preliminary analysis I picked these few finalists:

  • pholser/junit-quickcheck (Java)
  • NullVoxPopuli/aeonvera (Ruby)
  • SimonKagstrom/kcov (C++)
  • skinny-framework/skinny-framework (Scala)
  • paypal/squbs (Scala)
  • ben-manes/caffeine (Java)
  • coala/coala (Python)

I’m sorry for being late, but I need a few more days to analyze them properly and decide which one gets the prize. I will announce the winner on 21st of October, in six days. I will email everybody and publish my decision here.

18 Oct 2016: This is my analysis of that seven finalists. I tried to pay as much attention to each project as possible (they all are rather good).

pholser/junit-quickcheck (19K LoC, 80K HoC)

  • changes are not really traceable, for example this commit has no link to any GitHub issue, why was it made?
  • GitHub releases are there, but I didn’t find how exactly they are made, where is the release procedure? Also, they are not documented anyhow, there are no relase notes.
  • There are some utility classes, for example Lists, Items, and Sequences.
  • There are some -ER classes, for example Shrinker and SampleSizer (which has code-rich constructor).
  • The design of “generators” is not really object-oriented, I guess. They all are providers of procedures, not really “objects” in terms of OOP.
  • Score: 5

NullVoxPopuli/aeonvera (46K LoC, 835K HoC)

  • “Vendor” assets are right in the GitHub repo, it’s a bad practice (including jQuery)
  • It’s Ruby on Rails, which is MVC, which is not really OOP. And there is also an ORM with anemic model (in models/ dir). Aside from that, “serializers”, “services”, “validators”, etc. – not realy an OOP.
  • I didn’t find any GitHub releases
  • There is no official release procedure in the repo. I simply can’t understand how this product goes to production, the process is not automated (or the script is not in the repo). It’s a serious problem for a web app.
  • Aside from that, the app is definitely cleaner than many other similar RoR web apps. Good job.
  • Score: 4

SimonKagstrom/kcov (15K LoC, 50K HoC)

  • I didn’t find any static analysis, although there are many open source tools for C/C++.
  • Some pieces of code are too complex methods, for example at ptrace.cc, elf-parser.cc, or html-writer.cc.
  • There are some -ER classes, like “writers”, “verifiers”, and “parsers”, for example: AddressVerifier or DwarfParser.
  • Interfaces are prefixed with I, which is a bad practice in OOP, see reporter.hh.
  • There are some “utils”, which I would replace with classes (most of them): utils.hh.
  • Score: 4

skinny-framework/skinny-framework (44K LoC, 191K HoC)

  • It seems that the owner of the repo is making commits directly to the master branch, without any issues/pull requests, for example.
  • I didn’t find any static analysis.
  • I didn’t find code coverage control, although there are many unit tests.
  • There are utility classes, for example StringUtil, DateTimeUtil.
  • It’s MVC, which is an anti-pattern in OOP.
  • There is ORM, which also is an anti-pattern.
  • The complexity of the code sometimes is very high, for example this file: AssociationsFeature.scala
  • Score: 2

paypal/squbs (28K LoC, 163K HoC)

  • Where is static analysis?
  • Exception swallowing seems to be a common practice here, for example: here, here, here, etc.
  • There are utility classes, for example ConfigUtil.
  • Overall impression is that the complexity is rather high, for example these files are really hard to understand: ServiceRegistry.scala or UnicomplexBoot.scala.
  • Score: 3 (mostly due to complexity)

ben-manes/caffeine (47K LoC, 191K HoC)

  • Some files are very long, for example BoundedLocalCache.java (3300 LoC!). Aside from a few super long files, there are many .java files with 300+ LoC; it’s too much and complexity is high because of that. The code looks clean, but very complex.
  • There are some -ER classes, for example Weigher can easily be renamed to Weight. Same for “loaders” and “writers.”
  • Score: 4

coala/coala (15K LoC, 274K HoC)

  • This project was a finalist last year.
  • There are some -ER classes, like “parsers”, “collectors”, “importers.”
  • I still don’t see a consistent code formatting, it’s really strange to see so many spaces in front of some lines, placed there without any logic (at least I can’t see that logic, for example).
  • Some code is pure global, without any classes, like in Processing.py and in Collectors.py (maybe it’s inevitable, but still looks strange).
  • Some files are rather long, like Lynter.py or ConsoleInteraction.py.
  • Score: 5

I can’t find a strong single leader… Let me think about it.

23 Oct 2016: I didn’t find a single leader this year and the prize goes to two projects: pholser/junit-quickcheck and coala/coala ($2,048 to each one).

Congratulations to @pholser and @sils, the winners!

Here are your badges:

winner   winner

Put this code into GitHub README (replace ??? with your GitHub name in the URL):

<a href="https://www.yegor256.com/2015/10/17/award-2016.html">
  <img src="//www.yegor256.com/images/award/2016/winner-???.png"
  style="width:203px;height:45px;" alt='winner'/></a>

Thanks to everybody for participation! See you next year.

sixnines availability badge   GitHub stars