Tools

Synch Two Git Repositories Using Bundle File And USB-stick

November 16, 2016 git, Tools No comments

Imagine working on the same code base in two disconnected networks. How would you synchronize your repositories using rudimentary storage device, like a USB-stick?

Undeniably for such a synchronization there could be multiple solutions starting with very primitive manual copying of cloned repositories finishing with some specialized devices and synch processes.

I came up with something intermediate, until the situation with the setup of project changes.

git-bundle-synch-on-usb-device

Idea is very simple:

1. USB-sharing device, so that USB-stick can be shared with a press of a button (physical in this case)

2. git bash script that does the following:

  • Tries to connect to both repositories to identify which one is accessible
  • Fetches sources from available repository
  • Fetches sources from bundle file on USB-stick (git bundle file is like a zip file with history and all files)
  • Tries to merge these two folders with flag –allow-unrelated-histories so that history is completely preserved
  • If merge succeeds it pushes changes to available repository and recreates bundle
  • If merge fails, you would need to manually resolve conflicts and push

3. A task to trigger the synch script when USB-stick with bundle is connected (I do not have this one yet, but it is a next logical step)

If two repositories were available at the same time the same script (with modifications) could be used to synchronize them on schedule or trigger event.

Here is the code of the script:

 

I also make it available on github under MIT license. Hopefully it comes in handy.


No comments


Working with Excel files using EPPlus and XLSX.JS

February 29, 2016 C#, JavaScript, Opinion, QuickTip, Tools 1 comment

Need to quickly generate an Excel file on the server (.NET) with no headache or need to import  another Excel file in your JS client?

I can recommend two libraries for their simplicity and ease of use.

XLSX.JS

To smooth transitioning from Excel files to electronic handling of data we offered our users possibility of importing data. As our application is web based it meant some JS library to work with Excel files. A bit of complication was that our users over time developed a habit of having all kinds of modifications in their “custom” Excel files. So something that would allow us easily work with different formats was a preference.

XLSX.JS library available on GitHub proved to be a good choice. I could only imagine how much better it is over some monsters that would only work in IE. I think starting documentation is fairly good, so I will just go through some bits and pieces from our use case.

Setting up XLSX.JS and reading files is straight forward: npm or bower, include of file and you are ready to write XLSX.readFile('test.xlsx') or App.XLSX.read(excelBinaryContents, {type: 'binary'}).

Reading as binary is probably a better bet as it will work in IE, though you will have to write some code to implement FileReader.prototype.readAsBinaryString() in IE. You can have a look at our implementation of file-select component on gist.

Using XLSX in your JavaScript is fairly easy, though there might be some hiccups with parsing dates. See this gist.

EPPlus

We also have two use cases where we need to generate Excel file on the server. One was to generate some documentation for business rules so we can have it up to date and share with our users at all times. It was implemented as part of CI that would save a file to a file system. The other use case was downloading of business related data via web interface. These two were super easy to do with open source library called EPPlus.

You just add EPPlus through NuGet and start using (var excelPackage = new ExcelPackage(newFileInfo)). See the gist below. First file demonstrates how to operate with cells and the other one just shows how you can use streams to make file downloadable.

These two libraries really helped me to efficiently implement some of the Excel file business use cases.

Next time I will have to generate Excel file on server or read it on client I will most certainly use these two again.


1 comment


NDepend – a great tool to know more about your code

April 12, 2013 Tools 4 comments

I’ve always been working in enterprise projects. They usually involve tons of code and sophisticated extensive dependencies inside of code base.

On the other hand I’m keen on keeping code as clean and simple as possible. But how do you find problems quickly within huge set of solutions/projects/assemblies when lot of teams work on those? You have no idea who writes good and who writes bad code or where to look for potential bugs or ugly design.

When you have to analyze a lot of code there is no better static analysis tool than NDepend. There are quite a lot of tools which could perform some analysis here and there, but biggest advantage of NDepend is that everything is in one nice package.

Probably the most exciting feature of NDepend is Code Query LINQ. It allows you to write your own customized queries. Want to see all methods that have cyclomatic complexity higher than 5 and with more than 15 lines of code? Not a problem. Just run this:

from m in JustMyCode.Methods where 
  m.NbLinesOfCode > 15 &&
  m.CyclomaticComplexity > 5

select new { m, m.NbLinesOfCode, m.CyclomaticComplexity} 

As easy as that. Double click on a method and you are already editing it in Visual Studio.

I also like that tool provides analysis on different levels of details. On bigger scale you can identify problematic assemblies. For this purpose there are couple of features. One of them is “Abstractness vs. Instability”. It generates great image with distribution of your assemblies.

Image below was generated for one of projects I’m working on. If most of your assemblies fit into green zone than there is a good balance between abstractness vs. instability. I’m proud to say that code which I write always is in “green zone”.

image

I went through few of our projects and indeed projects with problematic code base is usually close or is in orange zone. It can be considered bad if you are in “Zone of Pain”. Scott Hanselman has nice post on this and NDepend in general called “Exiting The Zone of Pain – Static Analysis with NDepend”. Check it out.

Another high-scale features would be “dependency graph” and “dependency matrix”. While first one is something that comes with VS (well Ultimate), second one is more sophisticated and not found that frequently in other tools.

image

This feature gives you complex matrix of all dependencies all the way between assemblies. You can expand it to classes and even methods. On the picture above a dependency cycle is shown. What is great about NDepend is that it always explains in plan English why and what is shown. In this particular situation there was cycle dependency because compiler generated help class for linq query, so I ignored it as I cannot do much about it. But you never know, next time you might have a problem with your code.

When you run analysis NDepend would check against built-in code quality queries. What you get is something as below:

image

When you click on items on the right you get detailed list of matched classes/methods so you can go through them and verify if anything has to be changed. To be honest even that I’m very scrupulous when it comes to code quality, I discover a lot of new things for myself.

For example I got this in section “.NET Framework Usage”: Collection properties should be read only with an explanation from MSDN:

// A writable collection property allows a user to replace the collection with

// a completely different collection. A read-only property stops the collection

// from being replaced but still allows the individual members to be set.

// If replacing the collection is a goal, the preferred design pattern is to

// include a method to remove all the elements from the collection

// (with a call to the ICollection.Clear() method) and then re-populate the collection.

Of course it makes sense, but I just never thought about it. And there are many-many other things which you can learn from the tool only by using it.

During my working experience I started to think that people tend to create too many projects in VS when they finally feel comfortable with doing so. I like when there are as few assemblies as possible. Folders/namespaces are very good logical separation of code, there is no need in assemblies A, B and C if they always run in same application domain and depend on same stuff. NDepend team advices this and also reasons about it. You can read their “white-books”.

Other cool thing about NDepend is that it can be integrated with Visual Studio and Reflector. When you run NDepend it provides you with start page which proposes these add-ins for you. This is very handy.

It is natural that NDepend has console so you can run it from you scripts.

You can see complete list of NDepend features here: http://www.ndepend.com/Features.aspx

I’m afraid that my thoughts were somehow not well structured, I just picked some of the capabilities NDepend has and did not talk much about different use cases, but don’t worry. You will find answers to all your questions on web site: www.ndepend.com/

In my current company NDepend is integrated in build process, in my previous company I used it occasionally to see how we progress over time. Big companies usually take code quality seriously as smallest design flaws now could mean a lot of money leaks in future. Of course everything boils down to employees, but if they have right tools at their disposal you are protecting yourself. This great tool has a price that for sure doesn’t break the bank especially for company that claims to ship high quality software.


4 comments


IIS Logging hints: (Log Parser + Log Parser Studio + IIS Advanced Logging)

February 18, 2013 IIS, QuickTip, Tools 1 comment

Rather than starting this blog post with first excusing about me being not much into IIS. I will share very simple but extremely powerful toolset to be aware how your web application is living its life.

Of course any good application has exception handling and logging in place therefor you know when “shit happens” and possibly able to dig into it, find root cause and fix. Great! But what if you want to know more? What if you want to see some graphics showing distribution of users, their favourite features, pick hours, etc? To some extend Google Analytics could satisfy your needs. I use it for this blog and it is absolutely brilliant there.

You could have better tool in your hands? Plain simple. Plain power.

First. Enable your IIS Logging

To enable IIS logging use this boring step-by-step explanation. Or find this self-explanatory nice pictogram

image

in IIS manager. After enabling it in actions section your logs will be collected accordingly to configuration. (Say for application with high load you might want to change configuration to generate a new log file for each hour.)

Second. Analyse what’s inside

Well, first step could be considered as waste of keystrokes for some of you as you find it natural to have IIS logging enabled. But I also know that there are applications running for which no one ever collected any logs and have only preliminary idea how their applications are used and what is happening at all.

Here is bit of my story: We develop HTTP API used by various front-ends. Now we rewrite one of our heavily used endpoints. Therefor it was needed to know usage patterns (GET method parameters) and picks. I was able to quickly find everything out. The most time consuming was to download all those gigs of logs. But as soon as they are easily accessible, everything is piece of cake. Just continue reading…

You would need some tool to start analysing your logs.

Best fit would be Microsoft LogParser. Download it. It is console application which allows to query stuff with SQL-like queries. I said “stuff” because it is not only logs. Have a look on this LogParser architecture diagram:

logparser_architecture.gif

Output could be also charts (though there is dependency on MsOffice). For example this could be chart of non-200 status codes:

clip_image001

You can utilize this to to build nice reporting system with graphics to be sent to development team and/or managers.

Log Parser Studio

What if you want to quickly analyse something without bothering with console app?

There is Log Parser Studio – an excellent and handy tool.

image

When you just started it library of different queries is shown to you along with possibility to search among them. So you have a great starting point for your custom queries.

For example here is simple query I just used:

SELECT QUANTIZE(TO_TIMESTAMP(date, time), 60) AS Minute,
    sc-status,
    COUNT(*) AS Total
FROM *.log
WHERE (cs-uri-stem like '%SuspicionsEndpoint.svc%') and (sc-status = 500)
GROUP BY sc-status, Minute
ORDER BY Minute

I have found this page to be extremely useful as reference for functions in LogParser.

Be aware that LogParser doesn’t support everything available in SQL. When I tried to write something bit more complex like distinct count with grouping it complained. At least LogParser provided meaningful explanation so I knew that feature isn’t implemented. I had to use temporary file to achieve what I wanted.

Third (if you want more from IIS). Install Advanced Logging

You might be in situation when default logging fields are not enough for you:

image

For example, some token information is supplied in http header. No worries – Install IIS Advanced Logging extension. Now you look for this pictogram:

image

Setup and configuration is somewhat more complex.

Here is great step-by-step with pictures: http://www.iis.net/learn/extensions/advanced-logging-module/advanced-logging-for-iis-custom-logging

And here is mine shorter example. Say you want to include custom http request header:

  1. "Add Log Definition…”
  2. Configure it and include few common logging fields and save
  3. Navigate back to “Advanced Logging”, select your definition and hit “Edit Logging Fields…”
  4. Add logging field:
  5. image
  6. Use it in your definition
  7. Enable your definition for web site (if not yet enabled)

There you go. At this moment you can analyse even more information with Log Parser or nice Log Parser Studio.

Hope this post is of help for those who want to know more about their application!

Quick links:


1 comment


Sketching and prototyping with Firefox

July 3, 2010 Prototyping, Tools, UI No comments

I was thinking about what to post and I’ve got nice twits like this one:

@sbohlen I am quite impressed by this http://bit.ly/ci3fxn Its no #balsamiq but its still amazing (and free); runs in and OUT of your browser

So why not to try that thing. So I installed Pencil add-on.

Here is screenshot of my try:

Toolbar on the left is rich of different shape collections. Sadly I did not find one for UML diagrams, but they could be easily added and that is great.

So this is the simplest way to have light prototyping machine just in your browser.


No comments