Should I move from C++ to Python? ... Or another language?

In the company I work for, we do a lot of file-based transaction processing. The processing centers around the conversion of files between numerous formats to suit numerous systems in numerous companies.

The processing almost always involves an XML stage and can include a lot of text parsing, database lookups, data conversion and data validation.

Currently the programs performing all these tasks are written in C++ and they perform quite quickly all on one average server. I'm investigating the possibilities of using a more "modern" language that newer graduate programmers are more likely to be familiar with. (Correct memory allocation in C++ seems to causes problems with a lot of newer programmers these days)

Based on the brief information provided, would a language such as python provide the required functionality and performance, as well as addressing the memory allocation (and various other C++ related) problems which arise?

I like the idea of not needing to compile the programs each time we make a change. I understand that the interpreted languages probably wont hit the same performance we currently get.

Our systems are Linux based which also restrict some options.

Any comments on the functionality and performance available with Python or suggestions of alternative languages would be much appreciated.

From stackoverflow Jarod Elliott

• Python would probably remove most of the low level stuff that you use in your application. Memory allocation wouldn't be an issue anymore. Also, at least my university seems to be embracing Python as a programming language because students don't have to write all of that formal stuff to get started. Your only problem would be the performance part, as Python will likely never be as fast as a compiled C++ program.

  I would advise you to take a couple of weeks to get to know the programming languages that you're considering. I'd check out Ruby also. Maybe toy around with Haskell a bit?

  As I understand it Python seems well equipped for dealing with everything you're talking about. XML, database lookups, validation, parsing. It is usually a safe choice, not just because of the easy and fun programming experience, but if you're stuck there's an awesome community around the language who are just happy to help out.

  From deadtime

• I like the idea of not needing to compile the programs each time we make a change. I understand that the interpreted languages probably wont hit the same performance we currently get.

  This is the biggest issue; can you live with the performance hit. You could try to use Python and extending it with your current C++ modules for the performance heavy parts. Still, switching your entire system seems like a big effort if the only reason is the lack of C++ talent. Hiring people who know C++ seems like the cheaper option.

  gbjbaanb : poor programmers tend to be poor with all languages, so changing everything just to suit the numpties won't be a solution. I'd recommend teaching them how to be better instead, it'll pay off significantly. (and use STL and a nice XML lib - tinyXML is good)
  From mreggen

• Which is more important, quickly getting the programs to work, or getting the programs working quickly?

  If you're dealing with large numbers of large files then you may be better off staying in C++ and teaching your graduate programmers what a pointer is (!)

  Otherwise I'd strongly advise that you look at a scripting-based solution, because development in these, once you're up to speed, is so much faster. And a lot more fun, if we're honest, for most people at least.

  If the per-record processing load is not high, you may be surprised how little performance you lose: file IO will almost certainly be handled in a compiled (C) library, so the interpreter overhead may be relatively low. Worth trying, I'd suggest.

  Of the imperative languages, Perl is an obvious option, Python is popular and Ruby has a high profile (and probably cleaner OO features than the first two). Then there is the slightly more, er, esoteric realm of the functional languages, but I'm not qualified to comment on those.

  From Mike Woodhouse

• Another alternative is to embed Python in your C++ program. You could keep much of your application the same, and make calls out to Python for the pieces that change often, or need the flexibility that a scripting language provides.

  From the Python docs

  The previous chapters discussed how to extend Python, that is, how to extend the functionality of Python by attaching a library of C functions to it. It is also possible to do it the other way around: enrich your C/C++ application by embedding Python in it. Embedding provides your application with the ability to implement some of the functionality of your application in Python rather than C or C++. This can be used for many purposes; one example would be to allow users to tailor the application to their needs by writing some scripts in Python. You can also use it yourself if some of the functionality can be written in Python more easily.

  From Rob Thomas

• This is the biggest issue; can you live with the performance hit. Blockquote

  You can improve the performace using Psyco. And after all, thanks to Moor's law, performance isn't that big issue this days.

  From Samir

• Or should try to store your parsing rules on a database instead of leaving them hard-coded inside your code. As Ken Downs rightly quoted, minimize code, maximize data. This way you would not need to recompile everytime a tiny rule changes.

  From Mario Marinato -br-

• I hate to say this, but f you want something that your incoming developers are going to be familiar with, go with Java. Java is the language that most recent graduates will be most familiar with. You still have to compile, but compile times will be shorter than C++. It'll run on Linux and pretty much anywhere else. It's got a good garbage collector. It's pretty fast. And did I mention your developers will be familiar with it? No, it's not "cool" like Python, but it's a very tried-and-true language.

  Honestly, I doubt that you've got a lot of incoming developers who suck with C++ but would be awesome with Python anyway. The people who use Python well tend to be fine with manual memory management. The people who are bad with memory management actually tend to be bad with all languages.

  I do find it worrisome that you've got developers who are so bad with memory management that you want to switch languages. That's a sign indicating a problem, but I'm not sure that the problem is with the language.

  gbjbaanb : -1 for Java (doesn't really help the OP much at all), but +1 for "people who are bad with memory management tend to be bad with all languages".
  From Derek Park

• If you can get away with using Python, Ruby, Groovy or Perl vs. C++ you would be better off going with one of these higher level languages. Productivity will greatly increase. If you find that you need more performance then go with Java. Everyone should know at and use at least one dynamically typed language.

  From $pageUsers[$post.author].name

• should move to python that languange make all possible in networking, if you need faster move to c/c++

  From Gunslinger_

Is there some way of recycling a Crystal Reports dataset?

I'm trying to write a Crystal Report which has totals grouped in a different way to the main report. The only way I've been able to do this so far is to use a subreport for the totals, but it means having to hit the data source again to retrieve the same data, which seems like nonsense. Here's a simplified example:

       date   name   earnings   source          location
-----------------------------------------------------------
12-AUG-2008   Tom      $50.00   washing cars    uptown
12-AUG-2008   Dick    $100.00   washing cars    downtown     { main report }
12-AUG-2008   Harry    $75.00   mowing lawns    around town

                    total earnings for washing cars: $150.00 { subreport }
                    total earnings for mowing lawns:  $75.00

       date   name   earnings   source          location
-----------------------------------------------------------
13-AUG-2008   John     $95.00   dog walking     downtown
13-AUG-2008   Jane    $105.00   washing cars    around town  { main report }
13-AUG-2008   Dave     $65.00   mowing lawns    around town

                    total earnings for dog walking:   $95.00
                    total earnings for washing cars: $105.00 { subreport }
                    total earnings for mowing lawns:  $65.00

In this example, the main report is grouped by 'date', but the totals are grouped additionally by 'source'. I've looked up examples of using running totals, but they don't really do what I need. Isn't there some way of storing the result set and having both the main report and the subreport reference the same data?

From stackoverflow ninesided

• The only way I can think of doing this without a second run through the data would be by creating some formulas to do running totals per group. The problem I assume you are running into with the existing running totals is that they are intended to follow each of the groups that they are totaling. Since you seem to want the subtotals to follow after all of the 'raw' data this won't work.

  If you create your own formulas for each group that simply adds on the total from those rows matching the group you should be able to place them at the end of the report. The downside to this approach is that the resulting subtotals will not be dynamic in relationship to the groups. In other words if you had a new 'source' it would not show up in the subtotals until you added it or if you had no 'dog walking' data you would still have a subtotal for it.

  From N8g

• Can I just ask a couple of questions so I might be able to come up with an alternate solution... are you accessing this report from within an ASP.NET application?

  If so, does the report make the call on the database directly and "pull" the data or does it have data "pushed" to if from an underlying business layer?

  From lomaxx

• @lomaxx yep, it's accessed through a ASP.NET application and the report has a database stored procedure as its datasource. Currently, I have this stored procedure processing the data and writing the results to a table (which the subreport can use) as well as returning the result set, that way I don't have to process the data twice, just read it twice. I hope that makes sense!

  @N8g you've hit the nail on the head here:

  The problem I assume you are running into with the existing running totals is that they are intended to follow each of the groups that they are totaling. Since you seem to want the subtotals to follow after all of the 'raw' data this won't work.

  The problem with writing my own formulae is, as you pointed out, the source list is dynamic and could be different every time the report is run; hence the use of a subreport.

  From ninesided

• Hmm... as nice as it is to call the stored proc from the report and have it all contained in one location, however we found (like you) that you eventually hit a point where you can't get crystal to do what you want even tho the data is right there.

  We ended up introducing a business layer which sits under the report and rather than "pulling" data from the report we "push" the datasets to it and bind the data to the report. The advantage is that you can manipulate the data in code in datasets or objects before it reaches the report and then simply bind the data to the report.

  This article has a nice intro on how to setup pushing data to the reports. I understand that your time/business constraints may not allow you to do this, but if it's at all possible, I'd highly recommend it as it's meant we can remove all "coding" out of our reports and into managed code which is always a good thing.

  From lomaxx

Can I configure Visual Studio NOT to change StartUp Project everytime I open a file from one of the projects?

Let's say that there is a solution that contains two projects (Project1 and Project2).

Project1 is set as a StartUp Project (its name is displayed in a bold font). I double-click some file in Project2 to open it. The file opens, but something else happens too - Project2 gets set as a StartUp Project.

I tried to find an option in configuration to change it, but I found none.

Can this feature (though it's more like a bug to me) be disabled?

From stackoverflow Marek Grzenkowicz

• The way to select a startup project is described in Sara Ford's blog "Visual Studio Tip of the Day" (highly recommended). She has a post there about setting up StartUp projects. Essentially there are 2 ways, the easiest one being right-clicking on the desired project, and choosing "Set As StartUp Project". That prevents other projects from becoming the StartUp project, even if you click on one their files.

  From Lea Cohen

• Check your Visual Studio options for the following check box:
  Projects and Solutions - Build and Run - For new solutions use the currently selected project as the startup project.

  Uncheck that and see if the behavior changes.

  From kokos

Get list of domains on the network

Using the Windows API, how can I get a list of domains on my network?

From stackoverflow JosephStyons

• Answered my own question:

  Use the NetServerEnum function, passing in the SVTYPEDOMAIN_ENUM constant for the "servertype" argument.

  In Delphi, the code looks like this:

  <snip>
  type
    NET_API_STATUS = DWORD;
    PSERVER_INFO_100 = ^SERVER_INFO_100;
    SERVER_INFO_100 = packed record
      sv100_platform_id : DWORD;
      sv100_name        : PWideChar;
  end;
  
  function NetServerEnum(  //get a list of pcs on the network (same as DOS cmd "net view")
    const servername    : PWideChar;
    const level         : DWORD;
    const bufptr        : Pointer;
    const prefmaxlen    : DWORD;
    const entriesread   : PDWORD;
    const totalentries  : PDWORD;
    const servertype    : DWORD;
    const domain        : PWideChar;
    const resume_handle : PDWORD
  ) : NET_API_STATUS; stdcall; external 'netapi32.dll';
  
  function NetApiBufferFree(  //memory mgmt routine
    const Buffer : Pointer
  ) : NET_API_STATUS; stdcall; external 'netapi32.dll';
  
  const
    MAX_PREFERRED_LENGTH = DWORD(-1);
    NERR_Success = 0;
    SV_TYPE_ALL  = $FFFFFFFF;
    SV_TYPE_DOMAIN_ENUM = $80000000;
  
  
  function TNetwork.ComputersInDomain: TStringList;
  var
    pBuffer        : PSERVER_INFO_100;
    pWork          : PSERVER_INFO_100;
    dwEntriesRead  : DWORD;
    dwTotalEntries : DWORD;
    i              : integer;
    dwResult       : NET_API_STATUS;
  begin
    Result := TStringList.Create;
    Result.Clear;
  
    dwResult := NetServerEnum(nil,100,@pBuffer,MAX_PREFERRED_LENGTH,
                              @dwEntriesRead,@dwTotalEntries,SV_TYPE_DOMAIN_ENUM,
                              PWideChar(FDomainName),nil);
  
    if dwResult = NERR_SUCCESS then begin
      try
        pWork := pBuffer;
        for i := 1 to dwEntriesRead do begin
          Result.Add(pWork.sv100_name);
          inc(pWork);
        end;  //for i
      finally
        NetApiBufferFree(pBuffer);
      end;  //try-finally
    end  //if no error
    else begin
      raise Exception.Create('Error while retrieving computer list from domain ' +
                             FDomainName + #13#10 +
                             SysErrorMessage(dwResult));
    end;
  end;
  <snip>
  

  From JosephStyons

• You will need to use some LDAP queries

  Here is some code I have used in a previous script (it was taken off the net somewhere, and I've left in the copyright notices)

  ' This VBScript code gets the list of the domains contained in the 
  ' forest that the user running the script is logged into
  
  ' ---------------------------------------------------------------
  ' From the book "Active Directory Cookbook" by Robbie Allen
  ' Publisher: O'Reilly and Associates
  ' ISBN: 0-596-00466-4
  ' Book web site: http://rallenhome.com/books/adcookbook/code.html
  ' ---------------------------------------------------------------
  
  set objRootDSE = GetObject("LDAP://RootDSE")
  strADsPath =  "<GC://" & objRootDSE.Get("rootDomainNamingContext") & ">;"
  strFilter  = "(objectcategory=domainDNS);"
  strAttrs   = "name;"
  strScope   = "SubTree"
  
  set objConn = CreateObject("ADODB.Connection")
  objConn.Provider = "ADsDSOObject"
  objConn.Open "Active Directory Provider"
  set objRS = objConn.Execute(strADsPath & strFilter & strAttrs & strScope)
  objRS.MoveFirst
  while Not objRS.EOF
      Wscript.Echo objRS.Fields(0).Value
      objRS.MoveNext
  wend
  

  ---

  Also a C# version

  From Shane O'Grady

Delphi MDI Application and the titlebar of the MDI Children

I've got an MDI application written in Delphi 2006 which runs XP with the default theme. Is there a way of controlling the appearance of the MDI Children to avoid the large XP-style title bar on each window? I've tried setting the BorderStyle of the MDIChildren to bsSizeToolWin but they are still rendered as normal Forms.

From stackoverflow Marius

• I don't think there is; in my experience, MDI in Delphi is very strictly limited and controlled by its implementation in the VCL (and perhaps also by the Windows API?). For example, don't try hiding an MDI child (you'll get an exception if you try, and you'll have to jump through a couple of API hoops to work around that), or changing the way an MDI child's main menu is merged with the host form's.

  Given these limitations, perhaps you should reconsider why you'd like to have special title bars in the first place? I guess there are also good reasons why this MDI stuff is standardized --- your users might appreciate it :)

  (PS: nice to see a Delphi question around here!)

  From onnodb

• Thanks onnodb

  Unfortunately the client insists on MDI and the smaller title bar.

  I have worked out one way of doing it which is to hide the title bar by overriding the windows CreateParams and then create my own title bar (simple panel with some Mouse handling for moving). Works well enough so I think I might run it by the client and see if it will do...

  From Marius

• The way MDI works doesn't gel with what you're trying to do.

  If you need the "MDI" format, you should consider using either the built-in or a commercial docking package, and use the docking setup to mimic the MDI feel.

  In my Delphi apps, I frequently use TFrames and parent them to the main form, and maximizing them so they take up the client area. This gives you something similar to how Outlook looks. It goes a little something like this:

  TMyForm = class(TForm)
  private
    FCurrentModule : TFrame;
  public
    property CurrentModule : TFrame read FModule write SetCurrentModule;
  end;
  
  procedure TMyForm.SetCurrentModule(ACurrentModule : TFrame);
  begin
    if assigned(FCurrentModule) then
      FreeAndNil(FCurrentModule);  // You could cache this if you wanted
    FCurrentModule := ACurrentModule;
    if assigned(FCurrentModule) then
    begin
      FCurrentModule.Parent := Self;
      FCurrentModule.Align := alClient;
    end;
  end;
  

  To use it, you can simply do this:

  MyForm.CurrentModule := TSomeFrame.Create(nil);
  

  There is a good argument that you should use interfaces (creating an IModule interface or something) that you use. I often do that, but it's more complex than needed to explain the concept here.

  HTH

  From Tim Sullivan

• All your need - overload procedure CreateWindowHandle, like this:

  unit CHILDWIN;
  interface
  uses Windows, Classes, Graphics, Forms, Controls, StdCtrls;
  
  type
    TMDIChild = class(TForm)
    private
      { Private declarations }
    public
      { Public declarations }
      procedure CreateWindowHandle(const Params: TCreateParams); override;
    end;
  
  implementation
  
  {$R *.dfm}
  procedure TMDIChild.CreateWindowHandle(const Params: TCreateParams);
  begin
    inherited CreateWindowHandle(Params);
    SetWindowLong(Handle, GWL_EXSTYLE, WS_EX_TOOLWINDOW);
  end;
  end.
  

  From CoolMagic

• Thank you very much from CoolMagic

  From asgarkz

Getting accurate ticks from a timer in C#

I'm trying to rebuild an old metronome application that was originally written using MFC in C++ to be written in .NET using C#. One of the issues I'm running into is getting the timer to "tick" accurately enough.

For example, assuming an easy BPM (beats per minute) of 120, the timer should tick every .5 seconds (or 500 milliseconds). Using this as the basis for the ticks, however, isn't entirely accurate as .NET only guarantees that your timer will not tick before the elapsed time has passed.

Currently, to get around this for the same 120 BPM example used above, I am setting the ticks to something like 100 milliseconds and only playing the click sound on every 5th timer tick. This does improve the accuracy quite a bit, but if feels like a bit of a hack.

So, what is the best way to get accurate ticks? I know there are more timers available than the windows forms timer that is readily available in Visual Studio, but I'm not really familiar with them.

From stackoverflow jerhinesmith

• There are three timer classes called 'Timer' in .NET. It sounds like you're using the Windows Forms one, but actually you might find the System.Threading.Timer class more useful - but be careful because it calls back on a pool thread, so you can't directly interact with your form from the callback.

  Another approach might be to p/invoke to the Win32 multimedia timers - timeGetTime, timeSetPeriod, etc.

  A quick google found this, which might be useful http://www.codeproject.com/KB/miscctrl/lescsmultimediatimer.aspx

  'Multimedia' (timer) is the buzz-word to search for in this context.

  From Will Dean

• What is the C++ application using? You can always use the same thing or wrap the timer code from C++ into a C++/CLI class.

  From Coincoin

• Timer classes can start behaving strangely when the timer 'tick' event code is not finished executing by the time the next 'tick' occurs. One way to combat this is to disable the timer at the beginning of the tick event, then re-enable it at the end.

  However, this approach is not suitable in cases where the execution time of the 'tick' code is not acceptable error in the timing of the tick, since the timer will be disabled (not counting) during that time.

  If disabling the timer is an option, then you can also achieve the same effect by creating a separate thread that executes, sleeps for x milliseconds, executes, sleeps, etc...

  Wilhelm : But then you can only be sure that the thread sleeps at least x millioseconds; the thread scheduler does not certify that the thread will run at the exact millesond count

  Brad Barker : Right. I agree with what you said about not being able to pinpoint the time of the next tick. The point of what I am saying is you don't want to be executing tick event code from a previous tick when your next tick occurs.
  From Brad Barker

• System.Windows.Forms.Timer is limited to an accuracy of 55 milliseconds...

  From yazanpro

Is there any list datatype in MySQL stored procedures, or a way to emulate them?

I would like to create a stored procedure in MySQL that took a list as argument. For example, say that I would like to be able to set multiple tags for an item in one call, then what I want to do is to define a procedure that takes the ID of the item and a list of tags to set. However, I can't seem to find any way to do this, there is no list datatype, as far as I'm aware, but can it be emulated somehow? Could the list of tags be a comma-separated string, which can somehow be split and looped over?

How do you usually work with lists in MySQL stored procedures?

From stackoverflow Theo

• Hmmm, not sure if these will work specifically in a SP, but there are ENUM and SET datatypes in MySQL 5 which may do what you need. http://dev.mysql.com/doc/refman/5.0/en/enum.html http://dev.mysql.com/doc/refman/5.0/en/set.html

  From Shane O'Grady

• In my programming language of Choice, C#, I actually do this in the application itself because split() functions and loops are easier to program in C# then SQL, However!

  Perhaps you should look at SubString_Index() function.

  For example, the following would return google:

  SELECT SUBSTRING_INDEX(SUBSTRING_INDEX('www.google.com', '.', -2), '.', 1);
  

  From GateKiller

• Depending on how complicated you want to get, you can use a generic linking table. For one of my applications there are several reports where the user might pick, for instance a list of customers to run the report on rather than just a single customer from a combo box. I have a separate table with 2 fields:

  • UniqueID (guid)
  • ItemID

  The psuedo-code looks like this:

  GUID guid = GenerateGUID()
  try
    for each customer in customerList { INSERT(guid, customerId) }
    ExecuteSQLPocedure(guid)
    --the procedure can inner-join to the list table to get the list
  finally
    DELETE WHERE UniqueID=guid
  

  From Clyde

• This article has some good discussion on the problem of parsing an array to a stored procedure since stored procedures only allow valid table column data-types as parameters.

  There are some neat things you can do with the csv table type in mysql - that is if you are loading a flat file into the db.

  You could create a temporary table in the stored procedure, iterate over the csv list and insert it to the temp table, then create a cursor which selects the values from that table. This answer in the above mentioned thread shows a way of doing this.

  Generally I would split the array before I come to the database and then perform the query individually on each item.

  From roo