SciComp - Futures Volatility Surface Calibrator

Beware of the Python

There is a huge thing missing from Paul Wilmott's Big Book of Banking. Data. The dirty secret of much quant work is that sucking data out of databases, Bloomberg, Reuters, spreadsheets and files is between you and what you need to build to actually make some money. (Yes, people are making money, it’s just you). This can often be done with a little C++ or VBA, but often the data source has all sorts of garbage in it. This will include data you don’t need, and data that is simply wrong, or even corrupted. So the code to deal with it is going to be more complex than simple reading. To find the good, and filter out the bad data, may require a lot of business logic. A “bad” price may be a function of what the prices have been recently, or it may be an error such as when bond yields (typically 5-10%) are given, rather than the price (often around 100). Going from one to the other isn’t that hard, but if you’ve dealt with data you know that the source will sometimes just change for no reason. Corporate actions like stock splits will need to be lined up with time series of the price, and dates may be in the DD/MM/YYYY form or MM/DD/YYYY, or DD/JAN/YY or the number of milliseconds since Nassim Taleb was born. And yes, a given source will sometimes change with no warning, causing all sorts of fun. Sometimes the data will get rejected by the thing you want to put it in, and finding the error is helped by some degree of automation. This is best handled by a scripting language like Python, Perl, Ruby but I have seen UNIX Shell scripts based upon awk, sed and grep. This work is critical to your business unit, but it’s a dangerous are to get too deeply into. It’s very hard to be “excellent” at pumping datga around, but as the examples above show, it’s easy to be the person blamed for it going wrong. You are always playing catch up. It’s also work that is never finished, and you are the one they call to fix and upgrade it, and it’s far from unknown that this becomes your main work, sometimes all of it. The importance of the work gives a strong incentive for your boss to keep you doing it regardless of previous commitments to develop you at the firm. The fact that the toolset of 3rd party software and language is separate from “real” quant work just makes this worse. It’s harder for others to take your work, and many managers will see it as “trivial”. It is of course not very portable to other business units either. Search for jobs that say “Quant with data importing experience wanted” See how far you get. It’s nice to be useful, but it is not work that attracts the best bonuses, nor will you learn as much about your business. It is not just something that will slow down your progress, since this is basically IT work, there is a real danger that you will be sucked into the IT department, taking a big hit on your bonus, and crippling your career progression. You can’t easily avoid doing some of this, but you need to watch the % of your time spent doing it. If it seems to be trending up, then it’s unfortunately necessary to sit your boss down and talk this through. If the trend still goes up, you do really need to leave. Not today, but you probably have to leave before you find that you cannot leave.

Why we hate Computer Science Graduates

I’d just like to clarify my view on people who have studied Computer Science, since I have given the impression that I despise them. That’s because it’s true… At least in part. What we have here is not a directional term, but amplification under scrutiny. Whilst blogging and replying to posts I try to reflect the reality of succeeding in a quant career. That means saying nice warm fuzzy things like “C++ is good”, and harsh things like “CS grads who can’t do C++ are shambling morons”. That is the reality of the situation, indeed some managers at top tier places have told me that they don’t want to see the CVs of CompSci grads at all.
CompSci does not usually contain the sorts of maths most desired in this game, not least because the rules are typically set by applied mathematicians and physicists. But as I so often say, 60% of quant work is programming, and having an advanced qualification in a field sets an expectation that you can at least do the basics, even outside your speciality. Some Compsci courses don’t do C++, which is sad, and more than a little dimwitted, but in the real world you’re not likely to emerge as an ace C++ hacker from any undergrad programme, and only a tiny minority of PhDs require heavy duty C++ coding.
But what I expect on behalf of our clients is competence in C++ from those with formal CompSci qualifications. If someone who has done little or no programming for 5 years, having studied physics or economics can pick up a book and get to a baseline, then why exactly am I supposed to admire someone who has studied computers yet has not bothered to learn the language that is most important in this line of work ? Either too lazy, or they are too dumb to do C++. Failing to show basic effort and/or being weak at programming are not qualities that commend weaker CompSci Grads to us.
I am not ignorant of the difficulties of learning C++, having had to spend quite literally years of my life fighting the bloody thing. But C++ shows you to be tough enough to handle any crap that computer might throw at you, even if half your work is VBA and Matlab.

C++ in Quant Jobs

C++ is of course the dominant language in quant finance, utterly outclassing quiche languages like C# or Java.

There is of course an amount of Matlab, R, Mathematica etc, but they are of second order importance.

A large % of C++ in QF is essentially Fortran, and much code in any language is a barbarian horde of fiercely independent functions who only obey a strong leader who has slain his enemies and drunk their blood in a City wine bar.

"Academic" C++ is typically small compared to commercial code, and so you need OO, patterns et al to keep the horde from getting too bolshie. Commercial projects also go on for longer, some C++ code bases in banks is 20 years old, yes really, some of it's mine, so maintenance is a major issue, and re use is of course good.
But you will be asked questions on dangling pointers, stupidly complex type declarations, patterns, and various forms of broken code.
The correlation between real quant coding and what people get asked in interviews is thus not very good.

Data Analysis for Quant Finance

Lately we have seen a spate of good jobs looking for people who "understand" data. This does not have a formal specification, but seems important to several banks and hedge funds. They see people with skills in econometrics and signal processing, and that's fine, and can be searched for on CVs and recruitment databases. But it's not exactly what they are looking for, and it seems to be the sort "I know it when I see it" skill that exists merely to make the lives of headhunters like myself difficult. We (they) need people who can look at data series and intuitively grasp that there is a process there, or indeed that what they are seeing is pure noise, or caused by some defect in acquiring the data. Live tick prices are notorious for having bad and broken data points. There are many people who can apply various forms of regression and this is useful skill, but someone has to have this intuition as their lead skill, backed up by various bits of maths.
One idea we've been pursuing are fields like high energy physics where colossal streams of data are emitted, and require interpretation. Could just as well be market microstructure where you've studied price streams with a view to try and work out what people are doing, not just validating a theoretical model.
And yes, it needs to be that way round.
This sort of role demands that in your head, there is a good variety of "mechanical" or physical models for what is going on, and you have a vocabulary that allows you to choose between different classes of process. This is quite a distinct action from validating your finance or physical model against experimental data.
Presumably you have some software skills like Matlab, R, C++, SAS etc, like any functioning quant, but it is critical that you are not so dependant upon the rich functionality that large packages deliver that you let them do your thinking for you.
The firms concerned do not require formal qualifications in quant finance, indeed their expectation is often that the idea people for the role are working in a quite distinct field.
This is not a standard job spec, but the roles are in NY, Chicago and London, perhaps giving you an idea of the demand for this talent.
If this sounds like you, then contact me and we can informally discuss the way forward.