Recently, I wanted a way to iterate over two sequences in LINQ the same way you do in functional languages like python. It's most commonly called zip() and it works like this:
seq = [fn(a1, b1) for a1, b1 in zip(a, b)]
This applies a function fn() to each pair of variables pulled from a and b together. There is no direct way to do this in LINQ 3.5, but I came across MoreLinq which has an implementation of zip() and many other useful functions.
Recently, I had to implement the Excel function Rank() for a project. I came up with this templatized LINQ code:
Public Class ReversedComparer(Of T As IComparable)
Implements IComparer(Of T)
Public Function Compare(ByVal x As T, ByVal y As T) As Integer _
Implements System.Collections.Generic.IComparer(Of T).Compare
Return -x.CompareTo(y)
End Function
End Class
Private Shared Function BinarySearch(Of T As IComparable)(ByRef x() As T, _
ByVal val As T, _
ByVal comparer As IComparer(Of T)) As Integer
Dim lo As Integer = 0, hi As Integer = x.Count - 1
While lo <= hi
Dim mid As Integer = (hi + lo) \ 2
If comparer.Compare(val, x(mid)) <> 1 Then
hi = mid - 1
Else
lo = mid + 1
End If
End While
Return CInt(If(comparer.Compare(x(lo), val) = 0, lo, -1))
End Function
Public Shared Function Rank(ByRef X() As Double) As Integer()
Dim sorted() As Double = (From xx In X Order By xx Descending Select xx).ToArray()
Dim comparer As IComparer(Of Double) = New ReversedComparer(Of Double)
Return X.Select(Function(val) 1 + BinarySearch(Of Double)(sorted, val, comparer)).ToArray()
End Function
And it's pretty good as far as it goes. It takes a copy of the data in reverse-sorted order and then does a binary search to find the 0-based position in the array for each element in the original array. And that's fine, but I wanted to implement something more like my python code:
import collections def rank(arr): """ >>> a = list(range(10)) >>> print rank(a) [10, 9, 8, 7, 6, 5, 4, 3, 2, 1] >>> b = list(range(10)) >>> b.reverse() >>> print rank(b) [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] >>> c = [5] * 10 >>> print rank(c) [1, 1, 1, 1, 1, 1, 1, 1, 1, 1] >>> d = ([5] * 5) + ([1] * 5) >>> d [5, 5, 5, 5, 5, 1, 1, 1, 1, 1] >>> print rank(d) [1, 1, 1, 1, 1, 6, 6, 6, 6, 6] """ d = collections.defaultdict(list) for i, v in enumerate(arr): d[v].append(i) result = [0] * len(arr) i = 1 for k in sorted(d, reverse = True): for j in d[k]: result[j] = i i += len(d[k]) return result if __name__ == '__main__': import doctest doctest.testmod()
So I worked through the LINQ issues using the LinqPad interpreter and came up with this:
Function Rank(Of T)(Byref x() as T) as Integer()
Dim original_pos = x.Select(Function(xx, index) New With {.Val = xx, .Index = index}) _
.ToLookup(Function(xxx) xxx.Val)
Dim keys = original_pos.OrderByDescending(Function(yy) yy.Key)
Dim result(x.Count-1) as Integer
Dim i as Integer = 1
For Each item in keys
For Each v in item
result(v.Index) = i
Next
i = i + item.Count
Next
Return result
End Function
And I think that's pretty good code: templatized LINQ code that uses lambda functions and implements the algorithm as quickly as I know how.
A long time ago, I wrote CoWhereIs in C/C++. It was a small program to search the registry for where a COM server was to be found. Back in the mists of time, this was something you needed to do from time to time if you were a COM developer, as I was. And somewhere along the way, I stopped needing to do that and I lost the source code. But it was no big problem because I was no longer working with COM.
And now I find that I have some .NET code that implements user defined functions (UDFs) in Excel 2007 and it turns out that the integration between Excel and .NET is stuck in 2005: the code has to be exposed as a COM server that is installed with regasm on your machine. So now I need an installer and I need to test the installer to see if it is, in fact, calling regasm to patch the registry so that my .NET/COM server is actually available.
And so I found myself rewriting CoWhereIs to search the registry. This time, I've written it in python because the times have changed. Enjoy.
from __future__ import with_statement
from _winreg import *
import os.path
class MissingProgIDException(Exception):
pass
class MissingCLSIDException(Exception):
pass
def getCLSID(progID):
key = r'%s\CLSID' % progID
try:
with OpenKey(HKEY_CLASSES_ROOT, key) as h:
return QueryValue(h, "")
except WindowsError, err:
print r"Missing progID: HKEY_CLASSES_ROOT\%s" % key
raise MissingProgIDException()
def CoWhereIs(progID):
clsid = getCLSID(progID)
print "ProgID %s is CLSID %s" % (progID, clsid)
key = r'CLSID\%s\InProcServer32' % clsid
try:
with OpenKey(HKEY_CLASSES_ROOT, key) as h2:
location = QueryValueEx(h2, "CodeBase")
file_name = location[0].replace("file:///", "")
file_loc = file_name.replace(r'/', r'\\')
print file_loc, ": File exists?", os.path.exists(file_loc)
except WindowsError, err:
print r"Missing CLSID: HKEY_CLASSES_ROOT\%s" % key
raise MissingCLSIDException()
if __name__ == '__main__':
import sys
for arg in sys.argv[1:]:
try:
CoWhereIs(arg)
except:
pass
On a recent interview, I was asked: "Why doesn't C# have keywords for const-correctness?"
There are a few answers. First, C# does have constants and const class members. You can declare class members as const or readonly to get compile-time and run-time constants, respectively.
And it's even possible to write classes that are immutable, so that you cannot modify an existing one but have to make a new one, possibly based on an existing one. It's like strings:
s.replace("The", "An");
does not modify the original string. Instead, you have to take the return value:
s = s.replace("The", "An");
It won't come as a great surprise to hear that I favor making classes immutable wherever they can be, as a means of clarifying their usage and to make them safer.
But the real question is: why doesn't C# have const methods (methods where the code promises not to modify class members) or methods that respect const parameters? The nearest I have found to an answer on that is Anders Hejlsberg in an online interview:
Immutables
Bill Venners: In addition to being a C# and CLR construct, the value type is a general object-oriented programming concept. Another such concept is immutable types.
When I went to the first JavaOne back in 1996, everyone seemed to have one complaint about what they missed from C++ that Java didn't have. Different people had different complaints, but they all seemed to have at least one complaint. My complaint was const. I really liked what I could do with const in C++, though somehow I have gotten along in Java just fine without it.
Did you consider including support for the concept of immutable directly in C# and the CLR?
Anders Hejlsberg: There are two questions in here. With respect to immutability, it's tricky because what you're saying when you say something is immutable, is that from an external perspective, I cannot observe any mutation. That doesn't necessarily mean that it doesn't have a cache inside that makes it go more efficiently. It's just on the outside it looks immutable. That's hard for a compiler to figure out. We could certainly have a rule that says you can only modify the fields of this object in the constructor. And we could make certain usability guarantees. But it actually rules out some scenarios that are in use. So we haven't codified immutability as a hard guarantee, because it's a hard guarantee to make. The concept of an immutable object is very useful, but it's just up to the author to say that it's immutable.
Bill Venners: Immutability is part of the semantics of the class.
Anders Hejlsberg: Yes. With respect to const, it's interesting, because we hear that complaint all the time too: "Why don't you have const?" Implicit in the question is, "Why don't you have const that is enforced by the runtime?" That's really what people are asking, although they don't come out and say it that way.
The reason that const works in C++ is because you can cast it away. If you couldn't cast it away, then your world would suck. If you declare a method that takes a const Bla, you could pass it a non-const Bla. But if it's the other way around you can't. If you declare a method that takes a non-const Bla, you can't pass it a const Bla. So now you're stuck. So you gradually need a const version of everything that isn't const, and you end up with a shadow world. In C++ you get away with it, because as with anything in C++ it is purely optional whether you want this check or not. You can just whack the constness away if you don't like it.
The interviewees approach "Immutability [as] part of the semantics of the class." They consider this the relevant feature, not semantics of individual methods of a class.
Eric Gunnerson also considers const-correctness "a good idea in the small, but doesn't work too well in the large." Furthermore, he points out that it is hard to enforce across multiple languages.
There are also many entries on StackOverFlow.com on this question (searchable here), and the best answer provided is here:
I suspect there are some practical reasons, and some theoretical reasons:
- Should the constness apply to the object or the reference? If it's in the reference, should this be compile-time only, or as a bit within the reference itself? Can something else which has a non-const reference to the same object fiddle with it under the hood?
- Would you want to be able to cast it away as you can in C++? That doesn't sound very much like something you'd want on a managed platform... but what about all those times where it makes sense in C++?
- Syntax gets tricky (IMO) when you have more than one type involved in a declaration - think arrays, generics etc. It can become hard to work out exactly which bit is const.
- If you can't cast it away, everyone has to get it right. In other words, both the .NET framework types and any other 3rd party libraries you use all have to do the right thing, or you're left with nasty situations where your code can't do the right thing because of a subtle problem with constness.
There's a big one in terms of why it can't be supported now though:
- Backwards compatibility: there's no way all libraries would be correctly migrated to it, making it pretty much useless :(
I agree it would be useful to have some sort of constness indicator, but I can't see it happening, I'm afraid.
So, approaches you can use to get immutability at the class level:
The first two would support immutable method parameters -- you could not change them in the scope of the function and get side-effects. But C++-style const-correctness on methods is not available by language syntax.
A QuadTree is a spatial partitioning strategy used to make queries on relationships between 2D spatial data such as coordinates in a Geographic Information System (GIS), or the location of objects in a video game. For instance, you may need to know all of the objects within a region on a map, test weather objects are visible by a camera, or optimize a collision detection algorithm.
The QuadTree is so named because it recursively partitions regions into four parts, with leaf nodes containing references to the spatial objects. Querying the QuadTree is a function of traversing the tree nodes that intersect the query area.
The OctTree is the analogous structure used for 3 dimensional problems.
Are you...
- The type of developer who uses what works while keeping an eye out for a better way.
- You reach outside the mainstream to adopt the best of any community: Open Source, Agile, Java, Ruby, etc.
- You're not content with the status quo. Things can always be better expressed, more elegant and simple, more mutable, higher quality, etc.
- You know tools are great, but they only take you so far. It's the principles and knowledge that really matter. The best tools are those that embed the knowledge and encourage the principles (e.g. Resharper.)
In programming, there's no substitute for knowing your stuff. In versions 2 and 3, C# introduces new concepts such as lambda expressions and implicit typing that make the language more flexible and give you more power. Using Language INtegrated Query (LINQ)--also new in C# 3--you can interact with data of any type directly from C#. Simply put, mastering these features will make you a more valuable C# developer.
C# in Depth is designed to bring you to a new level of programming skill. It dives deeply into key C# topics--in particular the new ones. You'll learn to reuse algorithms in a type-safe way with C# 2 generics and expand the functionality of existing classes and interfaces using C# 3 extension methods. Tricky issues become clear in author Jon Skeet's crisp, easy-to-follow explanations and snappy, pragmatic examples. With this book under your belt, you will easily learn--and then master--new frameworks and platforms.
Duffy aims to give application, system, and library developers the tools and techniques needed to write efficient, safe code for multicore processors. This is important not only for the kinds of problems where concurrency is inherent and easily exploitableâsuch as server applications, compute-intensive image manipulation, financial analysis, simulations, and AI algorithmsâbut also for problems that can be speeded up using parallelism but require more effortâsuch as math libraries, sort routines, report generation, XML manipulation, and stream processing algorithms.
Concurrent Programming on Windows has four major sections: The first introduces concurrency at a high level, followed by a section that focuses on the fundamental platform features, inner workings, and API details. Next, there is a section that describes common patterns, best practices, algorithms, and data structures that emerge while writing concurrent software. The final section covers many of the common system-wide architectural and process concerns of concurrent programming.
If you need high performance, or a rich user experience, you should consider multithreading. With .NET you can develop stable and robust multithreaded applications with minimal effort. .NET Multithreading teaches the basics in an understandable and practical way. It then focuses on .NET's mechanisms for multithreading and shows how easy it can be to develop applications with them. The book covers several design approaches such as one-thread-one-class, the asynchronous design pattern, and using queues as buffers between threads. It explains best practices and how to avoid common multithreading pitfalls such as deadlock and race conditions.
This book is written for intermediate .NET developers who know C# or VB .NET, but are not assumed to have a background in multithreading. It is rich in examples that will help you understand the subject and produce multithreaded applications that have the power of C++ while keeping the ease and reliability of .NET.
NHibernate in Action begins by describing how to implement persistence in a layered .NET application. The book then quickly springs into action by introducing NHibernate through a classic "Hello World" example. It explains how to configure NHibernate to specify the mapping information between business objects and database tables, and then explores the internal architecture of NHibernate. A complete example application is progressively built with Agile methodologies in mind, which shows readers all kinds of entity and relationship mappings and how to perform CRUD operations. The book also covers advanced techniques like caching, concurrency access, and isolation levels. The Hibernate Query Language (HQL) and criteria query APIs are thoroughly detailed with optimization tips.
The last chapters of this book discuss various development scenarios, how to implement the layers of an NHibernate application (covering Windows and Web development), and which tools are available for these tasks. They also provide some solutions for data-binding objects to .NET GUI controls, integrating services, and interacting with components using DataSets. Finally, they explain how to build a complex application involving advanced session management and distributed transactions.
IronPython in Action offers a comprehensive, hands-on introduction to Microsoft's exciting new approach for programming the .NET framework. It approaches IronPython as a first class .NET language, fully integrated with the .NET environment, Visual Studio, and even the open-source Mono implementation. You'll learn how IronPython can be embedded as a ready-made scripting language into C# and VB.NET programs, used for writing full applications or for web development with ASP. Even better, you'll see how IronPython works in Silverlight for client-side web programming.
IronPython opens up exciting new possibilities. Because it's a dynamic language, it permits programming paradigms not easily available in VB and C#. In this book, authors Michael Foord and Christian Muirhead explore the world of functional programming, live introspection, dynamic typing and 'duck typing', metaprogramming, and more.
IronPython in Action explores these topics with examples, making use of the Python interactive console to explore the .NET framework with live objects. The expert authors provide a complete introduction for programmers to both the Python language and the power of the .NET framework. The book also shows how to extend IronPython with C#, extending C# and VB.NET applications with Python, using IronPython with .NET 3.0 and Powershell, IronPython as a Windows scripting tool, and much more.
LINQ, Language INtegrated Query, is a new extension to the Visual Basic and C# programming languages designed to simplify data queries and database interaction. It addreses O/R mapping issues by making query operations like SQL statements part of the programming language. Adding to its power, LINQ is extensible and can be used to query various data sources. It offers built-in support for querying in-memory collections like arrays or lists, XML, DataSets, and relational databases.
LINQ in Action is a fast-paced, comprehensive tutorial for professional developers who want to use LINQ. This book explores what can be done with LINQ, shows you how it works in an application, and addresses the emerging best practices. It presents the general purpose query facilities offered by LINQ in the upcoming C# 3.0 and VB.NET 9.0 languages. A running example introduces basic LINQ concepts. You'll then learn to query unstructured data using LINQ to XML and relational data with LINQ to SQL. Finally, you'll see how to extend LINQ for custom applications.
LINQ in Action will guide you along as you navigate this new world of lambda expressions, query operators, and expression trees. You'll also explore the new features of C# 3.0, VB.NET 9.0. The book is very practical, anchoring each new idea with running code.
Effective REST Services via .NET begins by discussing contemporary technologies and laying the groundwork for why REST services are so exciting and useful. It describes how they work and how both client and server-based code can access data using REST services. It details the URL, which is fundamental to accessing REST services. It then provides examples of the use of REST services in both desktop-based and Web-based applications.
The second half of the book delves into how to create REST services using existing .NET technologies, including ASP.NET (coupled with IIS 7.0), services hosted by the new ASP.NET model-view-controller (MVC) architecture, services hosted by the Windows Communication Foundation (WCF). The book finishes with coverage on exciting new technologies such as cloud computing (codename AZURE) and others.
Answer to a question from StackOverflow:
What are the best practices to do transactions in C# .Net 2.0. What are the classes that should be used? What are the pitfalls to look out for etc. All that commit and rollback stuff.
This article shows you how easy it is to use transactions with previous versions of .NET, and how you can use the TransactionScope object to auto-enlist database operations in a transaction. Other kinds of things, such as COM+ objects, can participate in transactions as well, but this article focuses on ADO.NET transactions.
PLINQ is a query execution engine that accepts any LINQ-to-Objects or LINQ-to-XML query and automatically utilizes multiple processors or cores for execution when they are available. The change in programming model is tiny, meaning you don't need to be a concurrency guru to use it. In fact, threads and locks won't even come up unless you really want to dive under the hood to understand how it all works. PLINQ is a key component of Parallel FX, the next generation of concurrency support in the Microsoft &174; .NET Framework.
Using technologies like PLINQ will become increasingly crucial to ensuring the scalability of software on future parallel microprocessor architectures. By utilizing LINQ at choice places throughout your applications today--such as where you have data- or compute-intensive operations that can be expressed as queries--you will ensure that those fragments of your programs continue to perform better when PLINQ becomes available and the machines running your application grow from 2 to 4 to 32 processors and beyond. And even if you only run that code on a single-processor machine, the overhead of PLINQ is typically so small that you won't notice a difference. In addition, the data parallel nature of PLINQ ensures your programs will continue to scale as the size of your data sets increases.
The Task Parallel Library (TPL) is designed to make it much easier to write managed code that can automatically use multiple processors. Using the library, you can conveniently express potential parallelism in existing sequential code, where the exposed parallel tasks will be run concurrently on all available processors. Usually this results in significant speedups.
TPL is being created as a collaborative effort by Microsoft &174; Research, the Microsoft Common Language Runtime (CLR) team, and the Parallel Computing Platform team. TPL is a major component of the Parallel FX library, the next generation of concurrency support for the Microsoft .NET Framework. Though it has not yet reached version 1.0, the first Parallel FX Community Tech Preview (CTP) will be available from MSDN &174; in Fall '07. Watch http://blogs.msdn.com/somasegar for details. TPL does not require any language extensions and works with the .NET Framework 3.5 and higher.
