These are chat archives for Microsoft/CodeContracts

29th
Jun 2015
Sam Harwell
@sharwell
Jun 29 2015 05:44
@SergeyTeplyakov It's late here. Can we sync up tomorrow to see what kind of schedule we are targeting for the updated release?
Sergey Teplyakov
@SergeyTeplyakov
Jun 29 2015 15:50
@sharwell Sure, lets discuss scheduling...
For ccrewriter I do have all fixes for all existing (i.e. opened) issues. I've talked to Francesco and he confirmed that my fix for #38 is workable.
The only thing that is required - is fix for VS2015 support itself (aside CC Editor extension that we can and should release separately). If I would be able to build the code with all latest fixes this week I would be able to chat with Mike and we'll create new installer for VS2015. I would suggest to share it via MVPs and via other insiders and in a week make it public (if no regression would be found).
Sam Harwell
@sharwell
Jun 29 2015 16:59
How does the testing situation look?
Sam Harwell
@sharwell
Jun 29 2015 17:12
My biggest points of confusion for a release are currently the following:
  1. What build/test cycle is used for a "complete build"
  2. What are all the solution configurations/platforms for? Ideally we can get this down to 1 debug configuration and 1 release configuration, and only one platform (probably "Mixed Platforms"). For the first release we don't need to prune it, but we do need to at least mention which one was used for the release.
Now that this is a community project, we should also share the preview with all participants in the project. We can attach the installer to the GitHub Releases page for it.
Sergey Teplyakov
@SergeyTeplyakov
Jun 29 2015 18:42

How does the testing situation look?

All unit tests are passing (for ccrewrite). All changes are reviewed by original author (by Mike). I'll test on our own codebase (we're using code contracts very extensively: few hundreds KLOC of code with a whole lot of contracts).

Other than that, I don't know what we can do.

As a bottom line, my idea was to use Mike for this release, because I would like to publish this new release to the old Code Contracts web-site (https://visualstudiogallery.msdn.microsoft.com/1ec7db13-3363-46c9-851f-1ce455f66970).

Now that this is a community project, we should also share the preview with all participants in the project. We can attach the installer to the GitHub Releases page for it

I'm ok with that.

Sam Harwell
@sharwell
Jun 29 2015 18:55
Sometimes Gitter just closes on me :\
Sergey Teplyakov
@SergeyTeplyakov
Jun 29 2015 18:55
:(
Sam Harwell
@sharwell
Jun 29 2015 18:56
How extensive are the unit tests? I haven't run coverage analysis on them.
I lean towards releasing a preview this week without #66, and attach the installer to the Code Contracts releases page on GitHub (tag the commit it was built from, and create a release for that tag)
That release would be marked as a pre-release, and clearly state that it includes initial support for compiling projects that use CC within VS 2015, but no editor extensions yet. Then we get the editor extensions in for the release right after that.
:question: Where are we on #6?
Sam Harwell
@sharwell
Jun 29 2015 19:02
@jaredpar Sorry to pull you in so abruptly, but can you point us to the right person who can walk us through using dotnet/codeformatter to do an initial formatting of a repository (or possibly restricted to a subset of files in a repository)? I'm specifically looking at #53. Depending on ease and reliability, I'd like to do this earlier rather than later so it's less intrusive.
Sergey Teplyakov
@SergeyTeplyakov
Jun 29 2015 20:44

How extensive are the unit tests? I haven't run coverage analysis on them.

They're limited. And believe me the code is super complicated, so code coverage doesn't mean anything here. And again, because the code is so complicated the only reasonable approach for short term - is to get a feedback from the original authors. What to do next? I'm not sure, because without Francesco it is almost impossible to add any significant changes to the CCCheck. CCRewrite is also super complicated (because it is based on IL rewriting) but code coverage is better.

Sam Harwell
@sharwell
Jun 29 2015 20:46
I understand. In that case, one thing we can do is implement a churn test for some large (public) project using Code Contracts, where we are just making sure that changes to Code Contracts doesn't change the analysis results for that project.
Sergey Teplyakov
@SergeyTeplyakov
Jun 29 2015 20:50
The problem is that CCCheck is not quite deterministic, so I don't know what to do with it. On the other hand, there is NO CHANGE in cccheck code base at all. And all the fixes fro ccrewrite (they're minimal) covered by tests.
BTW, I'm going to accept this pr: #60
Any objections?
Sam Harwell
@sharwell
Jun 29 2015 21:01
Not from me
I'm out for a few hours
Sergey Teplyakov
@SergeyTeplyakov
Jun 29 2015 21:05
ok. thanks.
Sam Harwell
@sharwell
Jun 29 2015 21:07
BTW - my questions about testing are primarily because I want to get some form of CI in place ASAP so changes which break the build/test cycle for other contributors are detected and reported before a pull request is merged.
In addition, the CI configuration file (e.g. appveyor.yml) make it very clear how new contributors can check out the code and compile it from source. If the CI can do it, so can they. :+1:
ttyl
Sergey Teplyakov
@SergeyTeplyakov
Jun 29 2015 21:13

I'm totally agree about CI and all this stuff. But as I mentioned, I fixed only ccrewrite tests, I don't know about ccchecks (they could be broken). And to fix ccrewrite tests I changed some of the ccrewriter logic (with confirmation from Mike and Francesco). I.e. we could have some issues for this automation. As you suggested we can disable some tests, but in this case our (small) code coverage would be even smaller and we still would not have a confidence that there is no regression.

So we would have kind of chicken and egg problem here: we don't have tests to check regression, but to fix tests we need to fix production code.

Long story short: I would love to have CI but not sure that it is possible to have it ASAP, just because it could take a while to get there.