Jamie's Space

JP

New Monitor

Today I got a new 24” Samsung T240 for my office at home. I’ve been working from home a lot recently and have really been missing my dual monitor setup from work, even though it is only my laptop with an old 17” LCD. Two monitors definitely make a big difference to my productivity when coding. I already had a 22” Samsung 226BW, which now looks thoroughly inadequate next to the 24” as you can see from the photo below.

My New Desk Arrangement

Once I got used to the larger monitor I had a pretty productive day though. The T240 is a decent monitor for developing on, judging from one day’s usage. Text is clear, and after a little tweaking the colour looked about right, enough for my purposes anyway. Not to mention it is a nice looking monitor. All in all, it was well worth the money I/the business spent on it.

Twitter, or How I Learned to Stop Worrying and Love Short Messages

Today I took a bold step, swallowed a chunk of my pride, and signed up for Twitter. I’m not usually a big fan of Social Networking, but Twitter is slightly different. For starters, I don’t think any of my friends are using it. Secondly, There’s no real “profile” like you get from Facebook or others.

I got the idea from Ayende’s blog, kindof. His blog is quite interesting to watch, because it is made up of lots of small updates. It almost seems like more of a note taking system than a blog sometimes, in a good way. For myself, I tend not to write blog posts because they take a little longer than I’m willing to give during the day to write, so Twitter’s 140 character limit could work for me. It’s just a trial anyway, if it doesn’t work out I’ll dump it.

So how I think I’ll work now is: Post short messages to Twitter like reminders, helpful hints, and status updates, and post longer information like information about side projects and interesting topics on my blog.

If anyone wants to follow me on twitter, here’s my page: http://twitter.com/jamiepenney. Be aware that there will be a bit more personal stuff there, my blog so far has just been technical but I’ll probably use Twitter for personal and work messages.

Ook Interpreter, Written in C# and F#

Another little short side project I worked on over the weekend was an interpreter for Ook, an odd little language “designed for Orangutans”. It only has a few syntax elements and 8 different statements so I figured it would be a fun use of my basic F# skills (I’ve been reading Fundamentals of F# and Expert F# on Safari Books through work). Using the example files on bluesorcerer.net as test files, I successfully got it running in half a day. I can’t say it is the fastest or most efficient interpreter for Ook, but it is my first interpreter so it was a learning experience. If you have any questions or comments on how I implemented it, please feel free to add them below.

Enough talk, here’s the GitHub project: http://github.com/jamiepenney/ook-sharp

You’ll need the latest F# CTP installed to open this project in Visual Studio 2008.

Viewing Dependencies Between Projects in Visual Studio

A while ago there was a question on Stack Overflow about how one could go about mapping the dependencies between Visual Studio projects and solutions (see here: http://stackoverflow.com/questions/471154/how-can-i-map-out-which-visual-studio-solutions-use-which-projects). At the time I had been doing a little bit of work with GraphViz, and suggested that it could be used for the diagramming part. The problem itself seemed kind of interesting, so I decided to implement a solution myself for kicks.

My tool takes a visual studio solution, and produces a PNG image showing the dependencies between the projects in that solution. Pretty basic stuff, but it involves:

  1. Parsing a Visual Studio .sln file.

  2. Parsing the .csproj files referenced in the .sln file. (I don’t think it is C# specific but have not tested it on other project types).

  3. Creating an in memory representation of the solution structure.

  4. Creating a graph in QuickGraph to represent that structure.

  5. Creating a dot file for GraphViz from that graph.

  6. Pass that file to GraphViz and display the output.

This results in something like this:

dv_output

The source code is here.

Updated: I’ve put this project on GitHub, and added Melle’s changes to it. You can find it here.

Feel free to do what you like with it, it only took me about 3-4 hours to get it all working. However if you actually use it as part of another project, please leave a comment.

You’ll need the latest version of GraphViz, which can be downloaded from here: http://graphviz.org/Download_windows.php. I’ve been using the development snapshot, but the stable release should be fine. When my tool first runs, it’ll ask you for the location of graphviz. You need to specify the location of dot.exe, which by default is in %Program Files%\GraphViz 2.21\bin.

Standard disclaimer: I’m not responsible for anything that goes wrong when using this thing. You have the source, you should check it out before running it. If it wipes your system and kills your firstborn child feel free to let me know, but don’t expect me to do anything about it.

TechEd 08 - Day 3

DEV375: Reactive Programming - Ivan Towlson

  • History of developing for windows clients

    • Win16/Win32 - Procedural style (message loops)

    • ActiveX/Visual Basic - Limited OO style, events + components

    • Windows Forms - Component oriented style

    • Common themes

      • Write code to copy data from model to control

      • Write code to detect changes in control data

      • Write code to copy data from control to model

      • Detect changes in model data or use gatekeeper

  • What is better about WPF/Silverlight?

    • Enhanced component model encapsulates common plumbing.

    • Designer-developer workflow needs a declarative programming style to work well.

    • Hard to write robust OO/procedural code against a lookless model.

  • Reactive programming is like binding values together. Think like how you can create formula’s in Excel - changing the value of one of the cells in a formula updates the result of the formula, and it keeps doing it.

  • Benefits

    • Remove plumbing code, no need to track changes in model and controls and copy data between

    • View/model relationship expressed in view instead of in controller/presenter code. Express the what, not the how.

    • Works well with the lookless approach - view reacts to model change, rather than the model manipulating the view.

  • .Net works on a notification system (INotifyPropertyChanged) - when something changes, dependent values re-evaluate themselves.

  • Issues

    • We need to react to changes to derived data -> use converters for this.

    • A reactive expression may produce unacceptable values (user input?). Solution is to use validation.

Bindable LINQ

  • How do you query an ObservableCollection? How do you observably query an ObservableCollection?

  • LINQ queries don’t provide change notification!

  • Bindable LINQ is a community project to enable reactive controls to back easily onto LINQ queries

  • Key API : AsBindable() - extension method on IEnumerable

SEC201: Do These Ten Things or Get Own3d - Steve Riley

1. Use precise terminology

  • Vulnerability - The problem

    • Code

    • Configuration

    • Circumvention - Security rules that are so draconian that they encourage people to actively try to get around it.

  • Threat - The person that carries out the attack

    • External

    • Internal

  • Exploit - the code that exploits the vulnerability

  • Exposure - The cost of the attack being carried out.

  • Risk

2. Accept that no one will like you

3. Think like a bad guy

People will not like you for this either.

4. Understand the science

  • Identity is not Authentication.

  • Encryption is not Integrity

  • Inspection is not Intent - Can’t tell intent by looking at someone’s stuff.

  • Secrecy is not Trust - just because you can make something a secret, doesn’t mean you can trust it.

  • People are not Technology - All the technology in the world will not solve your technology problem.

  • “Defence in depth” = “I am a parrot”

  • Stupidity is not Malice

  • Usability and Security are a tradeoff.

5. Protect your gear

  • Patch your stuff.

  • Use the firewall.

  • Don’t run as admin, or reduce number of admins.

  • Install anti-malware, or scan fileshares.

  • Don’t tweak.

  • Rebuild, don’t disinfect.

6. Swallow your pride

7. Reconsidering the laws

  • Law 1: If a bad guy can persuade you to run his program on your computer, it is not yours anymore.

  • Law 2: If a bad guy can alter the operating system on your computer, it’s not yours anymore.

  • Law 3: If a bad guy has unrestricted physical access to your computer, it’s not your computer anymore.

  • Law 4: If you allow a bad guy to upload programs to your website, it’s not your website anymore.

  • Law 5: Weak passwords trump strong security.

  • Law 6: A computer is only as secure as the administrator is trustworthy.

  • Law 7: Encrypted data is only as secure as the decryption key.

  • Law 8: An out of date is only marginally better worse than no virus scanner at all.

  • Law 9: Absolute anonymity isn’t practical, in real life or on the web.

  • Law 10: Technology is not a panacea.

8. Classify, and classify again.

  • Access

    • Principle of least privilege - others and yourself. If this principle interferes with your business process and is part of the OS, email Steve.
  • Data

    • Confidentiality, retention, recovery.
  • Trust

    • Functions, directions.

9. Don’t let your guard down.

  • People are always trying to sell you silver bullets/snake oil.

  • Trustworthy people will discuss the downsides of their software

10. Protection, not restriction.

TechEd 08 - Day 2

ARC202: I am not an Architect, I am an architect - Dr. Neil Roodyn

  • Why do most projects fail?

    • There are risks in developing software - laws, quality, misunderstood business needs, etc.

    • Unhappy customers.

  • History

    • Pioneering era - early computing. Software Development Process: Chaotic.

    • Stabilizing era - First demand for programmers,structure comes to development. SDP: Predictability. Based on engineering discipline.

    • Micro era - Drop in price of computing, more customer expectations.

    • Internet era - mid 90’s till present. Interconnectivity, change in pace of development. Customer expectations have outpaced developer abilities. SDP: Adaptive.

  • Predictability might be impossible in a creative field.

  • Requirements change in every project. “Change is Inevitable”.

  • Software needs to move away from the separation of architect and developer.

  • The majority of the cost of a system is usually the software development, and the major component of that is personal costs.

  • Craftsmanship is a different approach to software development.

    • Needs dedication - constant learning.

    • Duplication of software is low cost - this makes it more difficult to get revenue from your work.

    • Mentoring - Share experiences and knowledge.

    • Taking time out to think about things in the background - some of the best ideas are discovered while not actively thinking about the problem.

    • Accomplishment.

    • Need to set up processes so that developers can learn from their own and others mistakes.

  • Why promote the best coders into positions where they don’t write code?

  • Customers have a lot of choices - need to educate them on the tradeoff between cost and quality.

  • Happy developers create better quality code.

  • High quality software is possible - needs good supporting tools.

  • People over process.

SEC314: Secure Development Patterns - How not to screw yourself during development - Corneliu L. Tusnea

  • Distributed authentication systems work by verifying security tokens are valid, rather than verifying usernames and passwords on each service. The more times the password verification code has to be repeated the more chances there are for data leakage or failure.

  • Security should not be part of the code, it should be part of some external configuration - what user can use what services.

    • You still need to set attributes in code to determine what services to manage with security. This provides a better security model as there is only one place for look for specific security policy, and the code only needs to be referred to determine if it can be configured with security policy.
  • XSS protection is difficult to do yourself - the Microsoft security guys have written a library called AntiXSS, use that. HtmlEncode does not protect against XSS attacks. You should create a basic replacement for Label and Literal that AntiXSS encodes the text it displays and set up a TagMap to these new tags.

  • Validators are a source of duplication in a system.

  • Exceptions can leak information if used naively. A better method is to attach an enum to your exceptions, and to show a message based off of that. This gives you an easy way of internationalising error messages too.

WEB305: Pumping Iron: Dynamic Languages on .Net - Harry Pierson

  • Tradeoff between Type Safety and Flexibility

    • Rails ActiveRecord example - AR adds the column names of a table as properties on the model class at run time.
  • Real products are being shipped on IronPython - ResolverOne

  • Dynamic languages are very productive - good for initial work on greenfield projects.

  • Dynamic languages are not as fast as Static languages, so optimise by replacing slow parts of Iron(Ruby/Python) with C#. This is easy in the Iron* languages.

  • Dynamic Languages are easy to teach and learn.

    • Short on Ceremony - You don’t have to add extra stuff like package management or class creation to python if you don’t want to.
  • Formatting and whitespace matters. All Python code looks the same, so it is easy to start working with someone else’s code.

  • Dynamic languages are powerful.

    • AOP is easy in Python - it is possible to pass functions around as they are first level citizens, or you can use decorators to wrap functions.

    • You can create new types at runtime - example given was an XML to Python loader, which created Python objects from any given XML document.

  • Iron* languages are first class .Net citizens.

    • Interop with other .Net code. Easy to use C# or VB.Net code from Iron languages, but it is harder to go the other way. Standard .Net assemblies just work in Iron.
  • Iron* languages are embeddable by using the DLR.

  • Microsoft are working on integration with Visual Studio currently, due in 2009.

  • IronRuby and IronPython are both true Open Source.

Code Camp 2008

Thinking in WPF - Ivan Towlson

This talk was fantastic. My previous attempts at learning WPF failed miserably, but Ivan’s talk showed me that I was just thinking about programming a UI in WPF wrong. The programming model is very different to WinForms, and if you try to program in the old WinForms way you will have significant problems.

Test Driven Development and Dependency Injection - Robert Fonseca-Ensor

This was another excellent presentation. Robert covered the standard TDD stuff, but did a great job of explaining DI, as well as showing a short (~20 line) DI container that you can copy paste into any project. This tied in really well with the other talks.

ASP.Net MVC - Owen Evans

Owen covered most of the regular ASP.Net MVC stuff, but went into a little more detail than the last presentation I saw on it. There was a good discussion at the end about the difference between the MVC model and the WPF model of synchronising data between the View and the Model. The outcome of this discussion was that they are both useful, but for different areas. The stateless nature of HTTP means that the MVC pattern makes a lot of sense for websites, where as the use of data binding makes sense in a rich client application.

ASP.Net Dynamic Data - Scott Hanselman

This talk was a complete surprise for me. I hadn’t heard anything about Dynamic Data before this, and was very impressed with it. The gist of it is around scaffolding Linq entities. It has a bunch of built in logic for creating basic CRUD websites quickly.

The real surprise is the level of customisability - every single thing in this system is customisable. Every object is rendered by a template, but the automatic choices can all be overridden so if you don’t like that Category.Description renders with a TextArea, you can specify something else in code. The default templates are generated in your project, so you don’t like the default TextArea, you can change that template which changes the rendering for all of them.

Also Scott mentioned that this functionality will be making its way into the ASP.Net MVC project - once this happens, I will be completely sold. This is the sort of thing that would make CRUD style web development so easy without sacrificing customisabilty. Great stuff.

Silverlight - Jonas Follesco

An entertaining talk on some of the crazier things you can do with Silverlight. They have added support for using JavaScript objects in the C# code behind the silverlight application, and C# object in the JavaScript in the browser. He also demo’d a Silverlight - Javascript - Flash application using a webcam, which was a good end to the day.

He also covered Dependency Injection and TDD in silverlight.

TechEd 08 - Day 1

Figured I would write a little about each of the interesting sessions at TechEd, just as a note keeping exercise.

Keynote Speech

National’s John Key and Labour’s David Cunliffe talk about digital strategy.

National’s strategy: $1.5B over 6 years to a single utility for fiber to the home.

Labour’s strategy: $1B over 10 years to multiple providers.

Microsoft’s Amit —- talks about Software+Services

  • Local software extended with on demand computing services.

  • Small pieces loosely joined. Interoperability and standards become important.

  • Foster best of breed software and services.

  • Connecting devices together.

    • All devices seamlessly and securely integrate with each other.

    • Simple management of devices, software, and data.

    • Sharing of data between all of your devices.

  • Cloud computing, how do we split work between client and cloud.

  • Connected Business

    • Consistent UX across delivery and deployment options.

    • Common architecture and data models across deployments.

    • Flexibility and Adaptability in deployment.

WEB301: ASP.Net MVC - Should You Care? - Scott Hanselman

Very similar to Code Camp talk on ASP.Net MVC talk by Owen Evans. Scott talked mostly about the standard ASP.Net MVC features, but he is always entertaining to listen to so it was still worth watching.

ARC201: Moving Beyond Industrial Software - Harry Pierson

  • Architects are responsible for predicting and reacting to change.

  • Current Day:

    • Most IT departments are run like a factory. This is a bad model for creative work.

    • Change is happening now - traditional business models are falling apart.

  • Recommendations

    • Push control to the edge.

      • Centralized models of control cause bottlenecks.

      • Centralization as a technology no longer works.

      • Centralization slows you down - prevents marketplace agility.

      • “There is not one Microsoft anymore” - Steve Ballmer

      • Loose coupling between departments. Still need some decision making power at the centre, but not much. Central control over the budget is a good level.

    • Know when to ignore standards.

      • Choice between solving a business problem and adhering to a standard - solve the business problem!

      • The cost to maintain standards is not zero.

      • Adherence to standards costs as well.

      • Efficiency through Standardisation only occurs in a factory style environment - This is not IT!

      • What is the simplest thing that could work?

    • Empower users to solve their own problems.

      • IT people will never understand the business. If your business idea relies on good communication between IT and business people, you will probably fail.

      • Lack of marketplace within an enterprise means that there is no “natural selection” going on.

      • Build infrastructure and tools, not solutions.

      • Common, centralized infrastructure for business users to build their own solutions on. “If you want something done right, you’ve gotta do it yourself”

      • Common infrastructure costs can be amortized across the entire organization.

SEC306: Privacy - The Why, What, and How - Steve Riley

  • Data breaches in 3 1/2 Years: 227,120,380 in the US.

  • It is practically impossible to have a private face-to-face conversation with today’s invasive technologies.

  • Privacy: The right to be left alone.

  • Privacy laws and fines for data breaches have not caught up with technology - it is possible to expose your entire customer bases private data, but it is cheaper too pay the fine than fix the problem.

Steve’s session was more of a discussion. He went through a bunch of different scenarios, what was an acceptable levels of privacy invasion, what evil things it is possible to do with large amounts of user data. I don’t have much written down because I was too busy listening.

WEB302: ADO.Net Data Services - The zen of RESTfulness and the art of “Astoria” - Scott Hanselman

Astoria looks interesting - it is a framework for setting up REST web services from Entity Data Models. It looks pretty cool, you can then set up a service reference to this and use LINQ to query it. It really looks like Microsoft are setting all their stuff up to work with LINQ and enties now - very cool. The demo showed that you can use the same LINQ query on both a local database using LINQ to SQL, and by just changing the data context, you can query an Astoria web service.

As a side note, Astoria web services are just regular XML (in fact they return Atom), so it is possible to write your own implementation of this. You can set the expected data type (again, standard REST), so you can get back JSON if you want. This is important, because it allows them to use AJAX to pull this data back, and use the entities in the javascript using the actual property names. The data context has a few extra things in it, like the ability to batch requests. Again, these work in a standard REST style.

StringBuilder Performance Issues

We are working on a templating system, so as you can imagine we have to deal with the construction of massive strings of text. Conventional wisdom is to use a StringBuilder rather than a lot of String concatenation, but we wanted to test out the speed of StringBuilder to see if there were any issues that we may encounter. These might not be the best tests strictly speaking, but they show some interesting points that are worth thinking about. Here are our results:

Profiler_first_run

The first run was just to get the string builder code in memory.

Profiler_second_run_first_result_set

This is the same code as above, but hopefully the JIT has run over everything we are profiling so it should have less effect on our tests (if it has any at all in a debug build). Insert seems to be a touch faster but this could just be due to inaccuracies in the profiler though, as they are pretty small..

Profiler_second_run_second_result_set

Our last set of results. This time we took the ToString() call out of the loop and did it at the end. Look at the difference this makes to performance! Both Append and Insert perform nearly identically. The ToString() call itself is very quick, yet somehow it messes up the performance of Append and Insert drastically. Our application needs to periodically get the current state of the string builder, so we fall into the first category- where both Insert and Append perform terribly.

Test Scenario Time spent in loop (s)
Append and ToString 12.5000
Insert and ToString 12.2000
Append and ToString with preallocation 12.4000
Insert and ToString with preallocation 12.8000
Append without ToString 0.0092
Insert without ToString 0.0100
Append without ToString with preallocation 0.0083
Insert without ToString with preallocation 0.0103

Interesting results can come from performance profiling things - Why on earth does calling ToString() mess up the performance of StringBuilder, and is there any way around this?

UI Design, Unit Testing, and Related Pains

Background

Currently I am working on merging two of the steps in the Workbench part of ArchAngel. The final two steps of the generation process used to be the actual generation of the files you wanted, and the analysis to see which files needed to be merged. This was two screens, one in which the user picked the groups of files to generate, and the other which showed conflict resolution. Only after all conflicts had been resolved could the newly generated and merged files be copied into the user’s project directory.

UI Changes

This wasn’t as easy as it could be. There is no reason why the user needs to go through an additional screen to choose the files - why not just put that on the screen with the analysis so they could choose which files they wanted to keep? So Gareth merged the two screens, shifting the file choosing part to the tree view with checkboxes, and showing the results of the analysis as an icon next to the file in the tree.

Process Changes

One thing Gareth wanted to do was generate the files in the background. I set up a process whereby the files would be generated on a background thread, and if the user changed any options in the project, the files would be regenerated. Hopefully most of the files would be generated by the time the user actually got to the final generate and merge screen. We also do the analysis on the same thread once the generation is done.

Unit Testing

This process is really the meat of the ArchAngel Workbench - the majority of the processing happens here. Thus I wanted to make sure that I wasn’t breaking anything by implementing this background processing, as I had to make a number of changes to make it thread-safe. So I embarked on a quest to make this part of the system as easy to unit test as possible.

I started by completely removing any trace of GUI code from the generation code. This meant removing the background worker references, and replacing all of the code dealing with threading with a helper class I created. This class encapsulated progress reporting and task cancellation, so that we can mock these aspects during testing.

Mocking is a major focus of this process - I want to be able to mock out as much of the system as possible in order to make the unit tests more focused. The thing with mocking/faking the major components in the system is that we can write these mocks/fakes once, and reuse them in all of our tests. This is better than relying on resetting the state of the major components - this is prone to failure if those components change in future. Also it is difficult to prevent behaviour that relies on the underlying environment, such as writing to files and reading from settings files.

On Monday I will start working on creating some mocks for these components, and will add another post with my experiences of this.