Wednesday 2 July 2014

PyInstaller – Separating the executable from the ‘onedir’

Having had a quick play with py2exe, cx_Freeze and PyInstaller the one that worked for me least problematically was PyInstaller. I'm impressed with its ability to do the right thing out-of-the-box for a non-trivial app that uses PySide, PyOpenGL and NumPy among other 3rd-party libraries. My one real bugbear is that I don't really like either its 'onedir' or its 'onefile' modes, and there aren't any other options. As the name implies, 'onefile' bundles the whole mess into a single executable, which is sort of appealing, but this means that when the executable is run all those bundled files and directories have to be extracted somewhere, which adds a delay (which doesn't seem to be too bad on my developer box, but still). Furthermore, according to the manual, on Windows "… the files can't be cleaned up afterwards because Python does not call FreeLibrary", which means you have a bunch of stuff sitting in a temp directory (e.g. 'C:\\Users\\Me\\AppData\\Local\\Temp\\_MEI188722') with no real control over when it will be cleaned up (if ever). This kind of sets off my OCD, and I'd much rather have it be clear to both me and the end-user where all the files and directories relating to my application are (explicit is better than implicit). Which brings us to 'onedir' mode, which as the name implies shoves everything inside one directory, with the executable sitting alongside all the crap it depends on. This doesn't have to unpack anything from the executable when run, and there's no messing about with obscure temp directories, but the problem is that the 'one directory' just looks like a complete mess.
Again, this is probably just my OCD playing up, but I want to present a clean directory to my end users. Instead of this:


I want something more like this:

where 'lib' contains all the files and directories from the first screenshot that my app depends on. Unfortunately, there isn’t an easy way to get PyInstaller to do this. You can get most of the way there with only a little messing about though.

So the general idea is that we have a ‘lib’ directory (or whatever you want to call it) that contains the dependencies but that doesn’t contain the executable. If you just create one and move all the dependencies into it you’ll get errors at runtime because the dependencies can’t be found – nothing knows about your new lib directory. So the main thing you need to do is add your lib directory to the sys.path. You can’t do this in your main application module though because by the time that is run it is too late – the bootstrapping code will import stuff before your main script is run. So the way to do it is with a runtime hook. This is just a Python module; call it whatever you want and put something like the following in it:
import sys
import os
sys.path.append(os.path.join(os.path.dirname(sys.argv[0]), "lib"))
To have PyInstaller use this you can just pass the path to it as an argument to pyinstaller (alternatively you can specify it in your .spec file):
pyinstaller --onedir --windowed --runtime-hook=".\_pyinstaller_hooks\runtime_hooks\use_lib.py" …
Happily, this gets us most of the way there. Playing with this, by manually moving the built files about, I find that I can move everything into the lib directory except python27.dll (which makes sense), the ‘eggs’ directory and the PySide .pyd files. If you’re not using PySide then that last obviously won’t be an issue, so with a bit of luck just by amending the path in the runtime hook you should be able to make your ‘one dir’ distribution look like this:

which I’m sure you’ll agree is a lot nicer than the default. You’ll need to experiment to see whether the dependencies in your application can tolerate being moved into a separate directory from the executable; as noted PySide doesn’t like it (and I can’t find an easy way to fix this) but other things like NumPy seem to be perfectly happy. In case it’s not obvious, since PyInstaller isn’t set up to build this way out of the box you’ll need to move the files yourself – I just use a batch file that firstly calls pyinstaller to build the ‘one dir’ and then creates the lib directory and moves the relevant stuff into it. Finally, I’ve only tried this on Windows.

Update 15th August 2014

Here's the contents of my .spec file, as requested by Rishi Sharma:

# -*- mode: python -*-
a = Analysis(['..\\src\\my_app.py'],
             pathex=['C:\\dev\\python_projects\\my_app\\dist'],
             hiddenimports=[],
             hookspath=['.\\_pyinstaller_hooks\\additional_hooks'],
             runtime_hooks=['.\\_pyinstaller_hooks\\runtime_hooks\\use_lib.py']
)
pyz = PYZ(a.pure)
exe = EXE(pyz,
          a.scripts,
          exclude_binaries=True,
          name='my_app.exe',
          debug=False,
          strip=None,
          upx=True,
          console=True )
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=None,
               upx=True,
               name='my_app')

Note that this is just what pyinstaller automatically generates when I call it with the arguments shown above - I haven't tweaked it. The part highlighted shows how to set the hook and runtime hook paths - obviously you'll need to adjust it for your own paths. Hope this helps.

Sunday 28 July 2013

First model made with Cheetah3D

For a while I've been wanting to learn how to do 3D modelling, and after a discouraging start with Blender I decided to give Cheetah3D a go and have found I made much more progress. This is despite the fact there is a lot more in the way of free online tutorials for Blender. I don't want to knock Blender, I'm sure it's lovely when you get to grips with it, but really my experience with it, as a complete beginner, is that it makes 3D modelling appear to be something that's pretty hard to learn. With Cheetah3D, however, having only watched the paid-for tutorials by Andrew Hayworth, and some of the excellent free ones on YouTube by user NoVolume, I found that I could start to play around on my own and actually start to attempt modelling some real objects.

So here is a render of the first non-trivial 3D model I've ever made:


[there's a bigger version here

I started this after owning Cheetah3D for about a week. As I say, my only experience is watching some tutorials and messing about ineffectually with Blender for a few weeks. While competent 3D artists will no doubt be able to point out deficiencies, and while there are parts I'm not happy with and would go about differently if I had to do it again, I think it's a pretty decent first effort and shows that it's not that difficult to get started.
So if you're a beginner like me and you are wondering whether it's worth spending money on Cheetah3D when there is a free alternative in the form of Blender, then I'd recommend you go for it, unless of course you have a play with Blender and decide that you like it. Personally, I just find Cheetah3D more intuitive. I found that I could make progress just by choosing likely-sounding tools or options from the menu and playing with them, often with a quick read of the built-in help to clarify what some of the options do. Blender gives the impression that it has a lot of functionality, but doesn't expose it in an intuitive way. Having used Cheetah3D now and made some progress with it, I think I would probably get on better with Blender if I went back to it - I'd have a better idea of what to look for in terms of tools and options, although I think I'll stick with Cheetah3D for now.
It's worth saying that just watching this one video:

https://www.youtube.com/watch?v=M63wgHGXEdQ&feature=c4-overview-vl&list=PLD12BA4029778A06E

from NoVolume was enough to give me a very good start on my drill, and actually was really inspirational in showing the great results you can get with a few tools and in a short space of time, so thanks to him for posting that!

If anyone cares, the drill is this one:

which I chose mostly because this page:

http://www.homedepot.com/p/t/202511837?productId=202511837&storeId=10051&langId=-1&catalogId=10053&MERCH=REC-_-product-5-_-203358885-_-202511837-_-N#.UfXOvlOru1J

gives a 360 degree view of it, plus a nice big picture of the side view shown above, which was enough to get me some 'blueprint' images and get some kind of a view of the tricky parts. In the event, I only ended up with a nice side view, and a fairly crappy front view as my blueprints, so there are parts where I only had a sketchy idea of what it was supposed to look like and had to improvise, but on the whole it's pretty close.

Fun fact: if you search for images of drills you will see that almost all of them are in the orientation shown above (with the business end on the left and the handle on the right). Why is this? I can only guess it's because we live in a right-handed world and so (subconsciously) there is an expectation that the handle should be on the right. It makes me wonder if in the world of product imaging everyone knows that handheld items should be pictured oriented this way, or if they don't actually give it much thought but have all unconsciously ended up orienting them this way due to their innate right-handed bias.


Saturday 27 April 2013

Issues When Regularly Updating Windows Forms DataGridView

A couple of (for me) none-obvious things about using the Windows Forms DataGridView, in particular when you want to display data that is updated regularly. In my case, I have an IronPython application that is monitoring some messages that it has sniffed off the network, and I want to use a DataGridView to show some details of those messages and update it about once a second. Since I don't want to be doing much work on the GUI thread, I have a little thread whose job it is to get the new data that I want to be added to the DataTable that my DataGridView is bound to.

Don't Update the Data Source on the GUI Thread


Point 1: if you are gathering data for your DataGridView on another thread, don't use that thread to update the data source (such as DataTable) that your DataViewGrid is bound to. While anyone doing Windows Forms programming should know that you shouldn't update a Windows Forms Control from a non-GUI thread, it might not be completely obvious that you also can't update the data source that your Control is bound to from a non-GUI thread. Here's an example of what not to do - this is IronPython, but the same principle applies in other .NET languages:
class DataTrackerThread(threading.Thread):
    
    def __init__(self, main_form):
        super(DataTrackerThread, self).__init__()
        self.main_form = main_form
        self.empty_data_table = self.main_form.empty_data_table
            
    def run(self):
        try:
            # Assume that fresh_data gets populated with your new data, and is a list of
            # iterable objects, each one of which corresponds to a row we want merged
            # into our table:
            fresh_data = some_method_to_get_data()
            # First we'll put our new data into its own, clean table:
            new_dt = self.empty_data_table.Clone()
            for item in fresh_data:
                new_dt.Rows.Add(*[str(val) for val in item])
            # Now we want to merge our new table with the existing table - this
            # will cause the DataGridView bound to it to be updated:
            self.main_form.data_table.Merge(new_dt) # DON'T DO THIS!
            time.sleep(1)
        except Exception, e:
            logging.exception("Exception in DataTrackerThread.run")
Here, we have a problem because we've modified the main form's 'data_table' from a non-GUI thread, which really means that we're causing the DataGridView that is bound to the DataTable to change from a non-GUI thread. Interestingly, at least when I made this mistake, this caused a very specific and irritating problem: the vertical scrollbar of the DataGridView wouldn't display properly or allow scrolling (the horizontal scrollbar appeared to be fine). Here you can see the non-painting vertical scrollbar - this problem is caused entirely by udpating the DataGridView's DataTable from a non-GUI thread!
Furthermore, the entire form became very sluggish and would show the "... Not Responding" message in the title bar when I dragged it around.

So instead we need to update the DataTable in a thread-safe way, which we can do with the BeginInvoke mechanism. For my IronPython application, I added this method to the form containing the DataGridView and DataTable:
def invoke_merge_data_table(self, new_data_table):
    def do_it(new_data_table):
        self.data_table.Merge(new_data_table)
    delegate = CallTarget0(lambda: do_it(new_data_table))
    self.BeginInvoke(delegate)
and then all we need to do is replace this line:
self.main_form.data_table.Merge(new_dt)
with this:
self.main_form.invoke_merge_data_table(new_dt)
in our non-GUI thread.

Note that this problem is of course avoided if we don't have a non-GUI thread, and just do everything we need to do to update our data source in the GUI thread. However, if you're doing any non-trivial amount of work to update your data, and the chances are that you are, then you really shouldn't be doing it on the GUI thread, as you risk your application being unresponsive while it is doing the data gathering.

The DataGridView Auto-Scrolls when the DataTable is Updated


The second problem when frequently updating the DataGridView's underlying data source is that DataGridView automatically scrolls the currently selected row into view when its rows are modified. So if we need to scroll down to see rows that are currently outside the grid's viewable area, we find that the vertical scrollbar is effectively useless as it jumps back to the currently selected row when the grid's data is updated, which is every second in my case. Unfortunately, there isn't a simple way to turn this behaviour off. This seems to be a fairly well-attested problem - see here for example:

http://social.msdn.microsoft.com/Forums/en-US/csharpgeneral/thread/1a9ea234-a67b-42de-91b2-9658b170afd5
http://www.hightechtalks.com/dotnet-framework-winforms-controls/how-disable-auto-scroll-datagridview-540601.html
http://social.msdn.microsoft.com/forums/en-US/winformsdatacontrols/thread/9b78a1fa-207f-4cb6-8e1e-38579d2d0932

But I couldn't find a simple solution. The best suggestion appeared to be here:

http://stackoverflow.com/questions/1521396/how-do-i-prevent-a-datagridview-from-autoscrolling-when-the-datasource-changes

but unfortunately while he describes the solution he doesn't actually give an example. So for those having the same problem, here's an implementation of the solution suggested on that stackoverflow page, in IronPython. First, we need to add a method something like this to the form that owns the DataGridView:
def on_data_table_row_changed(self, sender, event):
    def do_it(restore_index):
        if restore_index > -1:
            self.myDataGridView.FirstDisplayedScrollingRowIndex = restore_index
    i = self.myDataGridView.FirstDisplayedScrollingRowIndex
    delegate = CallTarget0(lambda: do_it(i))
    self.BeginInvoke(delegate)
This is a handler for the DataTable's RowChanged event, so we need to assign it to that event like this in the form's constructor:
self.data_table.RowChanged += self.on_data_table_row_changed
All our handler does is use BeginInvoke to post a message to the form's message queue that when processed will execute our delegate which wraps our imaginatively named 'do_it' method. All 'do_it' does is manually set the scroll position according to the paramater passed to it. Note that when we handle the RowChanged event, the auto-scrolling hasn't yet occurred, so at the point when we set "i" to self.myDataGridView.FirstDisplayedScrollingRowIndex we have the 'correct' value for FirstDisplayedScrollingRowIndex - i.e. it hasn't been nobbled by auto-scrolling yet. That's why there's no point trying to change FirstDisplayedScrollingRowIndex in the on_data_table_row_changed handler itself. Instead we send a Windows Message (via 'BeginInvoke') that will get executed later, after auto-scrolling has nobbled the FirstDisplayedScrollingRowIndex value, and when this message gets handled 'do_it' gets called and sets FirstDisplayedScrollingRowIndex back to what it should be. Essentially the same idea is explained here (although he's not talking about a DataGridView) and that blog entry is referred to by the stackoverflow post that inspired my example.

This works pretty well, and when I tested it I could scroll vertically without the scrollbar jumping back up to the selected row. The only slight issue is that when we manually set the value of FirstDisplayedScrollingRowIndex the user's own manual scrolling with the vertical scrollbar is halted. Since I end up setting FirstDisplayedScrollingRowIndex about once a second, I can drag the vertical scrollbar for about a second, then it stops and I have to start dragging again. This is not ideal, but precisely how annoying it is will depend on how often your RowChanged handler gets called, which in turn depends how often your data source is updated. If you update, say, every 10 seconds I think you'd scarcely notice the scrollbar 'sticking', but if you update, say, every 0.1 seconds then it will be pretty annoying and make dragging the scrollbar handle not work very well at all. In any case, the 'jumping' of the scrollbar back to the selected row is fixed by this, so it's definitely an improvement, albeit not a perfect solution.

Tuesday 23 April 2013

Browsing for files and folders - a C# wrapper for SHBrowseForFolder

In an IronPython application I've been working on, I need to show the user a dialog that allows him or her to select either a file or a directory. I think there are legitimate use cases for this, but doing it on Windows turns out to be much more trouble than you might reasonably expect. Since this is an IronPython application, I have access to the Windows Forms API, but the long and short of it is that there isn't a standard Windows Forms component that gives you a good way of doing this. The FolderBrowserDialog will only let you browse for folders. The closest you can get is to do what this article suggests and use the OpenFileDialog with its FileName property given the magic value of "Folder Selection." plus some other settings, and this will sort of do what you want, but a) you have the string "Folder Selection." displayed in your 'File name' edit box, and b) if you click once on a file its name is put into the edit box, overwriting your magic "Folder Selection." string which means that you can't now select a folder. This seems unintuitive and kludgy to me. It is, however, the approach taken by WinMerge


so if it's good enough for them then it could well be good enough for you. But it does suck.

There is another approach which is to use the unmanaged SHBrowseForFolder function which shows essentially the same dialog as the WinForms FolderBrowserDialog, but unlike the FolderBrowserDialog it allows you to select either folders or files at the same time. Now I needed a .NET version of this since I'm using IronPython, and ideally a C# version since all the rest of my GUI is written in that language, and it turns out that there are some very good examples on the web already, in particular:

http://support.microsoft.com/default.aspx?scid=kb;[LN];306285
http://www.codeproject.com/Articles/3551/C-does-Shell-Part-1
and
http://dotnetzip.codeplex.com/SourceControl/changeset/view/29832#432677

However, there were two additional issues I wanted to address, and none of the examples I looked at dealt with them fully. Firstly, I wanted to be able to set the initially selected directory in the dialog when it is first shown, like the InitialDirectory property of the OpenFileDialog. Secondly, there is a bug in the dialog shown by SHBrowseForFolder, whereby on Windows 7 it doesn't always scroll to ensure the visibility of the selected item. This is quite well known, and doesn't seem likely to ever get fixed by Microsoft. It's quite annoying, and means that when I set an initial directory, then show the dialog, the folder I've selected is often not in view (it appears to be random as to whether it scrolls to it or not when the dialog is shown). There are workarounds suggested in several forums but I didn't find a working solution in C#. So I've put together my own simple wrapper for SHBrowseForFolder in C#, with an 'InitialDirectory' property and a workaround for the not-ensuring-visibility-of-selected-item bug. This is mostly based on the MS article mentioned above and uses the fix suggested here for the scrolling problem. Basically we just force the dialog to scroll the selected item into view whenever it changes. This results in the dialog visibly scrolling on its own, particularly when it's first shown, which isn't ideal, but is better than the selected item not being visible.

Anyway, here's what it looks like:


Not exactly a thing of beauty, but really when it comes to showing a file and folder browser on Windows you are left trying to decide which is the least-bad option.

And here's the code:

/*
 * This code is distributed under the terms and conditions of the MIT License.
 * 
 * http://opensource.org/licenses/mit-license.php
 * 
 * Copyright (c) 2013 John Dickinson
 * 
 * Permission is hereby granted, free of charge, to any person obtaining a copy 
 * of this software and associated documentation files (the "Software"), to deal 
 * in the Software without restriction, including without limitation the rights 
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 
 * copies of the Software, and to permit persons to whom the Software is 
 * furnished to do so, subject to the following conditions:
 * 
 * The above copyright notice and this permission notice shall be included in all 
 * copies or substantial portions of the Software.
 * 
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 
 * SOFTWARE.
 */

using System;
using System.Runtime.InteropServices;
using System.Text;
using System.Windows.Forms;

namespace FolderBrowserTest
{

    /// <summary>
    /// A dialog to browse for folders (and optionally files) that wraps the unmanaged SHBrowseForFolder function.
    /// </summary>
    /// <remarks>
    /// This is mostly based on the example given here: http://support.microsoft.com/default.aspx?scid=kb;[LN];306285
    /// with the main differences being that a) it allows the user to set an initial directory for the dialog, and
    /// b) it does something to address the bug in Win 7 (and above?) whereby the SHBrowseForFolder dialog won't
    /// always scroll to ensure visibility of the current selection.
    /// 
    /// Example usage:
    /// 
    /// FolderBrowser fb = new FolderBrowser();
    /// fb.Description = "Please select a file or folder below:";
    /// fb.IncludeFiles = true;
    /// fb.InitialDirectory = @"C:\Program Files\Windows Media Player";
    /// if (fb.ShowDialog() == DialogResult.OK)
    /// {
    ///     string userSelection = fb.SelectedPath;
    ///     ...
    ///     
    /// </remarks>
    public class FolderBrowser
    {

        // C# representation of the IMalloc interface.
        [InterfaceType(ComInterfaceType.InterfaceIsIUnknown), Guid("00000002-0000-0000-C000-000000000046")]
        private interface IMalloc
        {
            [PreserveSig]
            IntPtr Alloc([In] int cb);
            [PreserveSig]
            IntPtr Realloc([In] IntPtr pv, [In] int cb);
            [PreserveSig]
            void Free([In] IntPtr pv);
            [PreserveSig]
            int GetSize([In] IntPtr pv);
            [PreserveSig]
            int DidAlloc(IntPtr pv);
            [PreserveSig]
            void HeapMinimize();
        }

        // C# representation of struct containing scroll bar parameters
        [Serializable, StructLayout(LayoutKind.Sequential)]
        private struct SCROLLINFO
        {
            public uint cbSize;
            public uint fMask;
            public int nMin;
            public int nMax;
            public uint nPage;
            public int nPos;
            public int nTrackPos;
        }

        // Styles affecting the appearance and behaviour of the browser dialog. This is a subset of the styles 
        // available as we're not exposing the full list of options in this simple wrapper.
        // See http://msdn.microsoft.com/en-us/library/windows/desktop/bb773205%28v=vs.85%29.aspx for the complete
        // list.
        private enum BffStyles
        {
            ShowEditBox = 0x0010, // BIF_EDITBOX
            ValidateSelection = 0x0020, // BIF_VALIDATE
            NewDialogStyle = 0x0040, // BIF_NEWDIALOGSTYLE
            UsageHint = 0x0100, // BIF_UAHINT
            NoNewFolderButton = 0x0200, // BIF_NONEWFOLDERBUTTON
            IncludeFiles = 0x4000, // BIF_BROWSEINCLUDEFILES
        }

        // Delegate type used in BROWSEINFO.lpfn field.
        private delegate int BFFCALLBACK(IntPtr hwnd, uint uMsg, IntPtr lParam, IntPtr lpData);

        // Struct to pass parameters to the SHBrowseForFolder function.
        [StructLayout(LayoutKind.Sequential, Pack = 8)]
        private struct BROWSEINFO
        {
            public IntPtr hwndOwner;
            public IntPtr pidlRoot;
            public IntPtr pszDisplayName;
            [MarshalAs(UnmanagedType.LPTStr)]
            public string lpszTitle;
            public int ulFlags;
            [MarshalAs(UnmanagedType.FunctionPtr)]
            public BFFCALLBACK lpfn;
            public IntPtr lParam;
            public int iImage;
        }

        [DllImport("User32.DLL")]
        private static extern IntPtr GetActiveWindow();

        [DllImport("Shell32.DLL")]
        private static extern int SHGetMalloc(out IMalloc ppMalloc);

        [DllImport("Shell32.DLL")]
        private static extern int SHGetSpecialFolderLocation(IntPtr hwndOwner, int nFolder, out IntPtr ppidl);

        [DllImport("Shell32.DLL")]
        private static extern int SHGetPathFromIDList(IntPtr pidl, StringBuilder Path);

        [DllImport("Shell32.DLL", CharSet = CharSet.Auto)]
        private static extern IntPtr SHBrowseForFolder(ref BROWSEINFO bi);

        [DllImport("user32.dll")]
        [return: MarshalAs(UnmanagedType.Bool)]
        private static extern bool GetScrollInfo(IntPtr hwnd, int fnBar, ref SCROLLINFO lpsi);

        [DllImport("user32.dll", CharSet = CharSet.Auto)]
        private static extern IntPtr SendMessage(HandleRef hWnd, int msg, int wParam, string lParam);

        [DllImport("user32.dll", CharSet = CharSet.Auto)]
        private static extern IntPtr SendMessage(HandleRef hWnd, int msg, int wParam, IntPtr lParam);

        [DllImport("user32.dll")]
        private static extern bool PostMessage(IntPtr hWnd, uint Msg, int wParam, int lParam);

        [DllImport("user32.dll", SetLastError = true)]
        private static extern IntPtr FindWindowEx(IntPtr parentHandle, IntPtr childAfter, string className, string windowTitle);

        private static readonly int MAX_PATH = 260;

        /// <summary>
        /// Helper function that returns the IMalloc interface used by the shell.
        /// </summary>
        private static IMalloc GetSHMalloc()
        {
            IMalloc malloc;
            SHGetMalloc(out malloc);
            return malloc;
        }

        /// <summary>
        /// Enum of CSIDLs identifying standard shell folders.
        /// </summary>
        private enum FolderID
        {
            Desktop = 0x0000,
            Printers = 0x0004,
            MyDocuments = 0x0005,
            Favorites = 0x0006,
            Recent = 0x0008,
            SendTo = 0x0009,
            StartMenu = 0x000b,
            MyComputer = 0x0011,
            NetworkNeighborhood = 0x0012,
            Templates = 0x0015,
            MyPictures = 0x0027,
            NetAndDialUpConnections = 0x0031,
        }

        // Constants for sending and receiving messages in BrowseCallBackProc.
        private const int WM_USER = 0x400;
        private const int BFFM_INITIALIZED = 1;
        private const int BFFM_SELCHANGED = 2;
        private const int BFFM_SETSELECTIONW = WM_USER + 103;
        private const int BFFM_SETEXPANDED = WM_USER + 106;

        // Constants for sending messages to a Tree-View Control.
        private const int TV_FIRST = 0x1100;
        private const int TVM_GETNEXTITEM = (TV_FIRST + 10);
        private const int TVGN_ROOT = 0x0;
        private const int TVGN_CHILD = 0x4;
        private const int TVGN_NEXTVISIBLE = 0x6;
        private const int TVGN_CARET = 0x9;

        // Constants defining scroll bar parameters to set or retrieve.
        private const int SIF_RANGE = 0x1;
        private const int SIF_PAGE = 0x2;
        private const int SIF_POS = 0x4;

        // Identifies Vertical Scrollbar.
        private const int SB_VERT = 0x1;

        // Used for vertical scroll bar message.
        private const int SB_LINEUP = 0;
        private const int SB_LINEDOWN = 1;
        private const int WM_VSCROLL = 0x115;

        // Root node of the tree view.
        private FolderID rootLocation = FolderID.Desktop;

        /// <summary>
        /// Gets or sets the descriptive text displayed above the tree view control in the dialog box.
        /// </summary>
        public string Description { get; set; }

        /// <summary>
        /// Gets or sets the initial directory displayed by the dialog box.
        /// </summary>
        public string InitialDirectory { get; set; }

        /// <summary>
        /// Gets or sets the path selected by the user.
        /// </summary>
        public string SelectedPath { get; set; }

        /// <summary>
        /// Gets or sets whether the dialog box will use the new style.
        /// </summary>
        public bool NewStyle { get; set; }

        /// <summary>
        /// Gets or sets whether the dialog box can be used to select files as well as folders.
        /// </summary>
        public bool IncludeFiles { get; set; }

        /// <summary>
        /// Gets or sets whether to include an edit control in the dialog box that allows the user to type the name of an item.
        /// </summary>
        public bool ShowEditBox { get; set; }

        /// <summary>
        /// Gets or sets whether to include the New Folder button in the dialog box.
        /// </summary>
        public bool ShowNewFolderButton { get; set; }

        public FolderBrowser()
        {
            Description = "Please select a folder below:";
            InitialDirectory = String.Empty;
            SelectedPath = String.Empty;
            NewStyle = true;
            IncludeFiles = false;
            ShowEditBox = false;
            ShowNewFolderButton = true;
        }

        /// <summary>
        /// Creates flags for BROWSEINFO.ulFlags based on the values of boolean member properties.
        /// </summary>
        private int GetFlags()
        {
            int ret_val = 0;
            if (NewStyle) ret_val |= (int)BffStyles.NewDialogStyle;
            if (IncludeFiles) ret_val |= (int)BffStyles.IncludeFiles;
            if (!ShowNewFolderButton) ret_val |= (int)BffStyles.NoNewFolderButton;
            if (ShowEditBox) ret_val |= (int)BffStyles.ShowEditBox;
            return ret_val;
        }

        /// <summary>
        /// Shows the folder browser dialog box.
        /// </summary>
        public DialogResult ShowDialog()
        {
            return ShowDialog(null);
        }

        /// <summary>
        /// Shows the folder browser dialog box with the specified owner window.
        /// </summary>
        public DialogResult ShowDialog(IWin32Window owner)
        {
            IntPtr pidlRoot = IntPtr.Zero;

            // Get/find an owner HWND for this dialog.
            IntPtr hWndOwner;

            if (owner != null)
            {
                hWndOwner = owner.Handle;
            }
            else
            {
                hWndOwner = GetActiveWindow();
            }

            // Get the IDL for the specific startLocation.
            SHGetSpecialFolderLocation(hWndOwner, (int)rootLocation, out pidlRoot);

            if (pidlRoot == IntPtr.Zero)
            {
                return DialogResult.Cancel;
            }

            int flags = GetFlags();

            if ((flags & (int)BffStyles.NewDialogStyle) != 0)
            {
                if (System.Threading.ApartmentState.MTA == Application.OleRequired())
                    flags = flags & (~(int)BffStyles.NewDialogStyle);
            }

            IntPtr pidlRet = IntPtr.Zero;

            try
            {
                // Construct a BROWSEINFO.
                BROWSEINFO bi = new BROWSEINFO();
                IntPtr buffer = Marshal.AllocHGlobal(MAX_PATH);

                bi.pidlRoot = pidlRoot;
                bi.hwndOwner = hWndOwner;
                bi.pszDisplayName = buffer;
                bi.lpszTitle = Description;
                bi.ulFlags = flags;
                bi.lpfn = new BFFCALLBACK(FolderBrowserCallback);

                // Show the dialog.
                pidlRet = SHBrowseForFolder(ref bi);

                // Free the buffer you've allocated on the global heap.
                Marshal.FreeHGlobal(buffer);

                if (pidlRet == IntPtr.Zero)
                {
                    // User clicked Cancel.
                    return DialogResult.Cancel;
                }

                // Then retrieve the path from the IDList.
                StringBuilder sb = new StringBuilder(MAX_PATH);
                if (0 == SHGetPathFromIDList(pidlRet, sb))
                {
                    return DialogResult.Cancel;
                }

                // Convert to a string.
                SelectedPath = sb.ToString();
            }
            finally
            {
                IMalloc malloc = GetSHMalloc();
                malloc.Free(pidlRoot);

                if (pidlRet != IntPtr.Zero)
                {
                    malloc.Free(pidlRet);
                }
            }
            return DialogResult.OK;
        }

        private int FolderBrowserCallback(IntPtr hwnd, uint uMsg, IntPtr lParam, IntPtr lpData)
        {
            if (uMsg == BFFM_INITIALIZED && InitialDirectory != "")
            {
                // We get in here when the dialog is first displayed. If an initial directory
                // has been specified we will make this the selection now, and also make sure
                // that directory is expanded.
                HandleRef h = new HandleRef(null, hwnd);
                SendMessage(h, BFFM_SETSELECTIONW, 1, InitialDirectory);
                SendMessage(h, BFFM_SETEXPANDED, 1, InitialDirectory);
            }
            else if (uMsg == BFFM_SELCHANGED)
            {
                // We get in here whenever the selection in the dialog box changes. To cope with the bug in Win7 
                // (and above?) whereby the SHBrowseForFolder dialog won't always scroll the selection into view (see 
                // http://social.msdn.microsoft.com/Forums/en-US/vcgeneral/thread/a22b664e-cb30-44f4-bf77-b7a385de49f3/)
                // we follow the suggestion here: 
                // http://www.codeproject.com/Questions/179097/SHBrowseForFolder-and-SHGetPathFromIDList
                // to adjust the scroll position when the selection changes.
                IntPtr hbrowse = FindWindowEx(hwnd, IntPtr.Zero, "SHBrowseForFolder ShellNameSpace Control", null);
                IntPtr htree = FindWindowEx(hbrowse, IntPtr.Zero, "SysTreeView32", null);
                IntPtr htir = SendMessage(new HandleRef(null, htree), TVM_GETNEXTITEM, TVGN_ROOT, IntPtr.Zero);
                IntPtr htis = SendMessage(new HandleRef(null, htree), TVM_GETNEXTITEM, TVGN_CARET, IntPtr.Zero);
                IntPtr htic = SendMessage(new HandleRef(null, htree), TVM_GETNEXTITEM, TVGN_CHILD, htir);
                int count = 0;
                int pos = 0;
                for (; (int)htic != 0; htic = SendMessage(new HandleRef(null, htree), TVM_GETNEXTITEM, TVGN_NEXTVISIBLE, htic), count++)
                {
                    if (htis == htic)
                        pos = count;
                }
                SCROLLINFO si = new SCROLLINFO();
                si.cbSize = (uint)Marshal.SizeOf(si);
                si.fMask = SIF_POS | SIF_RANGE | SIF_PAGE;
                GetScrollInfo(htree, SB_VERT, ref si);
                si.nPage /= 2;
                if ((pos > (int)(si.nMin + si.nPage)) && (pos <= (int)(si.nMax - si.nMin - si.nPage)))
                {
                    si.nMax = si.nPos - si.nMin + (int)si.nPage;
                    for (; pos < si.nMax; pos++) PostMessage(htree, WM_VSCROLL, SB_LINEUP, 0);
                    for (; pos > si.nMax; pos--) PostMessage(htree, WM_VSCROLL, SB_LINEDOWN, 0);
                }
            }
            return 0;
        }

    }

}

Personally, I only find the non-scrolling bug to be a problem when the dialog is first shown, as I've just set an initial directory and this may end up not being visible. After that, whenever the selection changes it's because the user has changed it with the mouse or keyboard, so it's hard for the selection to go out of view then. On this basis, there is probably a good argument for only executing the code in the BFFM_SELCHANGED condition when the dialog is first shown, and it should be fairly trivial to add a flag to implement this as an exercise for the reader. Note however that when I've had a play at doing this, we actually go into the BFFM_SELCHANGED case three times when the dialog is first shown, and it's only on the third one that we actually scroll, since before this htic is always zero, so you need to cope with this - i.e. don't set your 'doneScrolling' flag (or whatever) until htic has been non-zero (or until you've gone into the BFFM_SELCHANGED three times).

Wednesday 27 February 2013

Subclassing QGraphicsEffect in PySide

I was recently looking at writing a QGraphicsEffect subclass, but could find surprisingly few (working) examples. Having now got it doing more or less what I wanted, I just wanted to share in case anyone else is looking for an example. All this does is draw a rectangular outline around a QGraphicsItem, like this:

Here's the code - Python 3 and using PySide, but will presumably work with PyQt if you change the imports:

"""
This code is distributed under the terms and conditions of the MIT License.

http://opensource.org/licenses/mit-license.php

Copyright (c) 2013 John Dickinson

Permission is hereby granted, free of charge, to any person obtaining a copy 
of this software and associated documentation files (the "Software"), to deal 
in the Software without restriction, including without limitation the rights 
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 
copies of the Software, and to permit persons to whom the Software is 
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all 
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 
SOFTWARE.
"""

from PySide.QtCore import Qt, QPoint, QRectF

from PySide.QtGui import QTransform, QPainter, QPen, QColor, QGraphicsEffect

DEFAULT_OUTLINE_WIDTH = 5
DEFAULT_OUTLINE_OFFSET = 1
DEFAULT_OUTLINE_COLOR = QColor(0, 0, 255)
DEFAULT_OUTLINE_LINESTYLE = Qt.SolidLine

class RectOutlineGraphicsEffect(QGraphicsEffect):
    
    def __init__(self, 
                 width=DEFAULT_OUTLINE_WIDTH,
                 offset=DEFAULT_OUTLINE_OFFSET,
                 color=DEFAULT_OUTLINE_COLOR,
                 style=DEFAULT_OUTLINE_LINESTYLE,
                 parent=None):
        QGraphicsEffect.__init__(self, parent)
        
        self.width = width
        self.offset = offset
        self.color = color
        self.style = style
        
    def boundingRectFor(self, rect):
        pad = self.width + self.offset
        return rect.adjusted(-pad, -pad, pad, pad)
        
    def draw(self, painter):
        
        offset = QPoint()
        if self.sourceIsPixmap():
            # No point in drawing in device coordinates (pixmap will be scaled 
            # anyway)
            pixmap = self.sourcePixmap(Qt.LogicalCoordinates, offset, 
                                       QGraphicsEffect.PadToTransparentBorder)
        else:
            # Draw pixmap in device coordinates to avoid pixmap scaling
            pixmap = self.sourcePixmap(Qt.DeviceCoordinates, offset, 
                                       QGraphicsEffect.PadToTransparentBorder)
            painter.setWorldTransform(QTransform())

        painter.setPen(QPen(self.color, self.width, self.style))
        if painter.testRenderHint(QPainter.Antialiasing):
            left_top_adjust = ((self.width - 1) // 2) + self.offset
            right_bottom_adjust = left_top_adjust + left_top_adjust
        else:
            left_top_adjust = (((self.width + 1) // 2) - 2) + self.offset
            right_bottom_adjust = ((self.width // 2) - 1) + left_top_adjust + \
                                  self.offset
        painter.drawRect(QRectF(offset.x() - left_top_adjust, 
                                offset.y() - left_top_adjust, 
                                pixmap.rect().width() + right_bottom_adjust, 
                                pixmap.rect().height() + right_bottom_adjust))
        
        painter.drawPixmap(offset, pixmap)
        

For your convenience I made a little application to test it:


Here's the code for the test app - it assumes the code for RectOutlineGraphicsEffect above is saved in a module called rect_outline_graphics_effect.py:

#!/usr/bin/env python

"""
This code is distributed under the terms and conditions of the MIT License.

http://opensource.org/licenses/mit-license.php

Copyright (c) 2013 John Dickinson

Permission is hereby granted, free of charge, to any person obtaining a copy 
of this software and associated documentation files (the "Software"), to deal 
in the Software without restriction, including without limitation the rights 
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 
copies of the Software, and to permit persons to whom the Software is 
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all 
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 
SOFTWARE.
"""

import sys

from PySide.QtCore import Qt, QRectF

from PySide.QtGui import (QPainter, QPen, QApplication, QMainWindow, 
                          QHBoxLayout, QVBoxLayout, QWidget, QLabel, 
                          QPushButton, QSpinBox, QComboBox, QGraphicsView, 
                          QGraphicsScene, QGraphicsItem, QGraphicsRectItem, 
                          QColorDialog)

import rect_outline_graphics_effect as cust_effect
       
class TestGraphicsScene(QGraphicsScene):

    def __init__(self, parent=None):
        QGraphicsScene.__init__(self, parent)
       
        self.effect_width = cust_effect.DEFAULT_OUTLINE_WIDTH
        self.effect_offset = cust_effect.DEFAULT_OUTLINE_OFFSET
        self.effect_color = cust_effect.DEFAULT_OUTLINE_COLOR
        self.effect_style = cust_effect.DEFAULT_OUTLINE_LINESTYLE
       
        self.rect = None

    def mousePressEvent(self, e):
        if e.button() == Qt.RightButton:
            # Draw a rect of size 1 initially. We'll embiggen it in 
            # mouseMoveEvent
            self.rect = QGraphicsRectItem(QRectF(e.scenePos().x(), 
                                                 e.scenePos().y(), 1, 1))
            self.rect.setFlag(QGraphicsItem.ItemIsMovable, True)
            self.rect.setFlag(QGraphicsItem.ItemIsSelectable, True)
            effect = cust_effect.RectOutlineGraphicsEffect(self.effect_width,
                                                           self.effect_offset,
                                                           self.effect_color,
                                                           self.effect_style,
                                                           self)
            self.rect.setGraphicsEffect(effect)
            self.rect.setPen(QPen(Qt.black, 1))
            self.addItem(self.rect)
        elif e.button() == Qt.LeftButton:
            QGraphicsScene.mousePressEvent(self, e)

    def mouseMoveEvent(self, e):
        if self.rect and (e.buttons() & Qt.RightButton):
            self.rect.setRect(QRectF(self.rect.rect().x(), 
                                     self.rect.rect().y(), 
                                     e.scenePos().x() - self.rect.rect().x(), 
                                     e.scenePos().y() - self.rect.rect().y()))
        QGraphicsScene.mouseMoveEvent(self, e)
       
    def outline_width_changed(self, value):
        self.effect_width = value
        for item in self.items():
            item.graphicsEffect().width = value
            item.graphicsEffect().updateBoundingRect()
           
    def outline_offset_changed(self, value):
        self.effect_offset = value
        for item in self.items():
            item.graphicsEffect().offset = value
            item.graphicsEffect().update()
           
    def outline_color_changed(self, color):
        self.effect_color = color
        for item in self.items():
            item.graphicsEffect().color = color
            item.graphicsEffect().update()
           
    def outline_style_changed(self, value):
        selection = [Qt.SolidLine, Qt.DashLine, Qt.DotLine, Qt.DashDotLine, 
                     Qt.DashDotDotLine][value]
        self.effect_style = selection
        for item in self.items():
            item.graphicsEffect().style = selection
            item.graphicsEffect().update()
           
class MainWindow(QMainWindow):

    def __init__(self):
        super(MainWindow, self).__init__()

        self.scene = TestGraphicsScene()
        self.scene.setSceneRect(QRectF(0, 0, 600, 600))
       
        self.view = QGraphicsView(self.scene)
        self.view.setRenderHint(QPainter.Antialiasing)
       
        self.width_spin = QSpinBox()
        self.width_spin.setMinimum(1)
        self.width_spin.setValue(cust_effect.DEFAULT_OUTLINE_WIDTH)
        self.width_spin.valueChanged.connect(self.scene.outline_width_changed)
       
        self.offset_spin = QSpinBox()
        self.offset_spin.setMinimum(0)
        self.offset_spin.setValue(cust_effect.DEFAULT_OUTLINE_OFFSET)
        self.offset_spin.valueChanged.connect(
                                            self.scene.outline_offset_changed)
       
        self.color_button = QPushButton("")
        self.color_button.setStyleSheet("background-color: rgb(%s, %s, %s);" % 
                                    (cust_effect.DEFAULT_OUTLINE_COLOR.red(),
                                     cust_effect.DEFAULT_OUTLINE_COLOR.green(),
                                     cust_effect.DEFAULT_OUTLINE_COLOR.blue()))
        self.color_button.clicked.connect(self.effect_color_select)
       
        self.line_style_combo = QComboBox()
        self.line_style_combo.addItems(["SolidLine", "DashLine", "DotLine",
                                        "DashDotLine", "DashDotDotLine"])
        self.line_style_combo.setCurrentIndex(0)
        self.line_style_combo.activated.connect(
                                            self.scene.outline_style_changed)
       
        control_layout = QHBoxLayout()
        control_layout.addWidget(QLabel("Width:"))
        control_layout.addWidget(self.width_spin)
        control_layout.addSpacing(10)
        control_layout.addWidget(QLabel("Offset:"))
        control_layout.addWidget(self.offset_spin)
        control_layout.addSpacing(10)
        control_layout.addWidget(QLabel("Color:"))
        control_layout.addWidget(self.color_button)
        control_layout.addSpacing(10)
        control_layout.addWidget(QLabel("Style:"))
        control_layout.addWidget(self.line_style_combo)
        control_layout.addStretch()

        main_layout = QVBoxLayout()
        main_layout.addWidget(QLabel(
                        "Drag with right mouse button to draw rectangles. "
                        "Select and drag them around with the left mouse "
                        "button"))
        main_layout.addWidget(self.view)
        main_layout.addLayout(control_layout)

        self.widget = QWidget()
        self.widget.setLayout(main_layout)

        self.setCentralWidget(self.widget)
        self.setWindowTitle("Example Graphics Effect Subclass")
       
    def effect_color_select(self):
        color = QColorDialog.getColor(self.scene.effect_color)
        if color.isValid():
            self.color_button.setStyleSheet(
                                        "background-color: rgb(%s, %s, %s);" % 
                                                            (color.red(),
                                                             color.green(),
                                                             color.blue()))
            self.scene.outline_color_changed(color)
       
if __name__ == '__main__':
    app = QApplication(sys.argv)
    frame = MainWindow()
    frame.show()
    sys.exit(app.exec_())


You can see that the RectOutlineGraphicsEffect has member variables for the width, color and line-style of the pen it is drawn with, as well as a member that controls the 'offset', the distance between the QGraphicsItem and the effect. Using the test application above you can change all of these dynamically to see what it looks like. Draw rectangles by dragging with the right mouse button pressed. You can then select and move those rectangles by clicking them with the left mouse button and dragging them.

Things to Note


What you definitely don't want to do when writing a QGraphicsEffect subclass is paint outside the bounds. If you do, your (re)painting won't work properly and you'll get odd effects. For example, when you drag your QGraphicsItem you'll see something like this:

which probably isn't what you want.

The notes on QGraphicsEffect are fairly brief, and may leave you with some doubts, so I'll try to explain what you need to do with regard to painting and the bounds.

Firstly, bear in mind that all QGraphicsEffect gives you is a QPixmap of the QGraphicsItem the effect is being applied to - you don't have access to the QGraphicsItem itself. You are just drawing on top of a bitmap (or manipulating the bitmap itself), and this limits what you can usefully do. One of the things you can do, and what my example above does, is paint outside of the bounds of the underlying QGraphicsItem. This is because QGraphicsEffect has its own bounds which can be bigger than the pixmap of the QGraphicsItem that you are drawing on. To specify what those bounds are you need to override boundingRectFor. All you are doing here is returning a rectangle that says what the bounds of your QGraphicsEffect are. Here's mine:

def boundingRectFor(self, rect):
    pad = self.width + self.offset
    return rect.adjusted(-pad, -pad, pad, pad)


The rect passed in here is the size of the QPixmap derived from the underlying QGraphicsItem we are going to apply an effect to, so if you just return it then you're saying that the bounds of your QGraphicsEffect are the same as the pixmap of the QGraphicsItem. In my case, I'm drawing around the QGraphicsItem so I need to specify bigger bounds, enough to include the width of my pen and the offset between the QGraphicsItem and the outline I'm drawing.

The documentation for the draw method of QGraphicsEffect gives you a stub of C++ code to get you started:

MyGraphicsEffect::draw(QPainter *painter)
 {
     ...
     QPoint offset;
     if (sourceIsPixmap()) {
         // No point in drawing in device coordinates (pixmap will be scaled anyways).
         const QPixmap pixmap = sourcePixmap(Qt::LogicalCoordinates, &offset);
         ...
         painter->drawPixmap(offset, pixmap);
     } else {
         // Draw pixmap in device coordinates to avoid pixmap scaling;
         const QPixmap pixmap = sourcePixmap(Qt::DeviceCoordinates, &offset);
         painter->setWorldTransform(QTransform());
         ...
         painter->drawPixmap(offset, pixmap);
     }
     ...
 }

which in our Python example has become:

def draw(self, painter):
        
        offset = QPoint()
        if self.sourceIsPixmap():
            # No point in drawing in device coordinates (pixmap will be scaled 
            # anyway)
            pixmap = self.sourcePixmap(Qt.LogicalCoordinates, offset, 
                                       QGraphicsEffect.PadToTransparentBorder)
        else:
            # Draw pixmap in device coordinates to avoid pixmap scaling
            pixmap = self.sourcePixmap(Qt.DeviceCoordinates, offset, 
                                       QGraphicsEffect.PadToTransparentBorder)
            painter.setWorldTransform(QTransform())

I'm not sure if we ever actually get inside the self.sourceIsPixmap() condition, but it doesn't matter for the purposes of our example since we're applying the effect to QGraphicsRectItem objects. I leave it as an exercise for the reader to see what happens when applying it to a pixmap.

Anyway, the key thing to understand here is the call to sourcePixmap. The first parameter is the coordinate system to use for describing the pixmap the method returns. I've just followed the C++ example here. The second parameter, offset, is a QPoint that gets set with the coordinates of the top left of the pixmap returned by the method (in the coordinate system specified in the first parameter). In other words the QGraphicsItem that we're going to apply the effect to has been turned into a bitmap, the method returns us that bitmap, and offset gets the top left of that bitmap. (Apologies for having a member self.offset, and this local offset - they aren't related. This local offset is just using the name the sourcePixmap method gives to this parameter, while the member self.offset is literally the offset from the QGraphicsItem to the outline, and I'm following the usage of QGraphicsDropShadowEffect which has a self.offset that does something similar). The bitmap may or may not include some padding around the underlying QGraphicsItem, which brings us to our third parameter.

If the third parameter is set to QGraphicsEffect.NoPad then the QPixmap returned will coincide with the bounds of the underlying QGraphicsItem. If it's set to QGraphicsEffect.PadToEffectiveBoundingRect then the pixmap returned will be the size of the rectangle you've specified in boundingRectFor (which could be significantly bigger than the underlying QGraphicsItem). Finally, if the third parameter is set to QGraphicsEffect.PadToTransparentBorder one pixel of padding will be added around the pixmap returned. In all cases, both the width and height of the returned pixmap, and offset, which describes its top left in our coordinate system, will be affected by what you choose as the third parameter.

Note that if you are writing your own QGraphicsEffect subclass and want to see the effect of the three different padding flags then you can just paint a rectangle at the coordinate specified by offset and with the width and height of the pixmap. In other words, a rectangle that shows the unmodified position and dimensions of the pixmap returned by sourcePixmap:

def draw(self, painter):
    offset = QPoint()
    if self.sourceIsPixmap():
        pixmap = self.sourcePixmap(Qt.LogicalCoordinates, offset, 
                                   QGraphicsEffect.PadToTransparentBorder)
    else:
        pixmap = self.sourcePixmap(Qt.DeviceCoordinates, offset, 
                                   QGraphicsEffect.PadToTransparentBorder)
        painter.setWorldTransform(QTransform())
    
    painter.setRenderHint(QPainter.Antialiasing, False)
    painter.setPen(QPen(Qt.red, 1, Qt.SolidLine))
    painter.drawRect(QRectF(offset.x(), 
                            offset.y(), 
                            pixmap.rect().width(), 
                            pixmap.rect().height()))
    painter.drawPixmap(offset, pixmap) 

And here's what we get when we do this with the three different flags:

In this example, the pixmap is significantly larger when using PadToEffectiveBoundingRect because my boundingRectFor method adds 6 pixels of padding.

To recap, if you paint outside of the bounds of your QGraphicsEffect you will get problems, and the bounds are defined by your implementation of boundingRectFor. The size of the pixmap returned by sourcePixmap may or may not be the size of the rectangle defining your QGraphicsEffect's bounds (it will be if you use PadToEffectiveBoundingRect). So depending which flag you pass to sourcePixmap, you may or may not be able to paint outside of the bounds of the pixmap returned by it, but in all cases if you paint outside of the bounds defined in boundingRectFor you will get problems. If you look at my example at the top of this post, you'll see I am painting a larger rectangle than the one returned from sourcePixmap ("offset.x() - left_top_adjust" etc.), but it doesn't cause a problem because I'm using PadToTransparentBorder and I'm still within the bounds described by boundingRectFor.

The rest of the draw method is just normal painting stuff, except that you have to remember to draw the pixmap of your underlying QGraphicsItem at the end if you want to see it.

Problems


If you look at the code where I work out how big my outline rectangle needs to be:

if painter.testRenderHint(QPainter.Antialiasing):
    left_top_adjust = ((self.width - 1) // 2) + self.offset
    right_bottom_adjust = left_top_adjust + left_top_adjust
else:
    left_top_adjust = (((self.width + 1) // 2) - 2) + self.offset
    right_bottom_adjust = ((self.width // 2) - 1) + left_top_adjust + \
                  self.offset
painter.drawRect(QRectF(offset.x() - left_top_adjust, 
                        offset.y() - left_top_adjust, 
                        pixmap.rect().width() + right_bottom_adjust, 
                        pixmap.rect().height() + right_bottom_adjust))

There's obviously quite a lot of messing about and we have to behave differently depending on whether we're painting with anti-aliasing. What I want is simply to have my outline painted with the width given by the effect's self.width member, at the offset from the QGraphicsItem given by the self.offset member. The rectangle we draw has to be adjusted by some values from the dimensions of the pixmap returned by sourcePixmap, although how much by depends on which padding flag we use. I happen to be using PadToTransparentBorder but whichever one we use we still have to make adjustments. Anti-aliasing in particular changes the width of the pen (a pen of width 5 will be painted 6 pixels wide), so we need to accommodate it.

What I've found though is that there is a limit to what you can do to paint the outline in the right place. In particular, if I set the width of the QGraphicsRectItem's pen to 2 (instead of 1 as it is in the example code above):

self.rect.setGraphicsEffect(effect)
self.rect.setPen(QPen(Qt.black, 2))

then I don't get the offset painted how I want:


Here, the offset is set to zero, so the blue outline should be touching the black rectangle of the QGraphicsRectItem. While this is true for the top and left of the outline, it's not true for the bottom and right. The problem is that the change in width of the pen that we draw the QGraphicsRectItem with has broken the adjustments I make in draw. While I have a special case to cope with anti-aliasing, I don't have any logic to cope with whether the QGraphicsRectItem's pen-width is odd or even, which is what I would need to fix this problem. The difficulty with that though is that the QGraphicsEffect subclass doesn't have any access to the QGraphicsItem it is being applied to. All it has is a pixmap of that QGraphicsItem, which doesn't 'know' what pen it was drawn with.

There is of course a work-around for this, which is to add my own QGraphicsItem member to my QGraphicsEffect subclass, and have it set in __init__,so that my QGraphicsEffect would be initialised something like:

self.rect = QGraphicsRectItem(QRectF(e.scenePos().x(), 
                                     e.scenePos().y(), 1, 1))
...
effect = cust_effect.RectOutlineGraphicsEffect(self.rect, self.effect_width, 
                                               self.effect_offset, self.effect_color,
                                               self.effect_style, self)

I could then refer to the QGraphicsRectItem in my draw method, check its pen-width and adjust the dimensions of my outline rectangle accordingly. Partly I haven't done this just to keep the example simple, but there's also an ugliness to coupling my QGraphicsEffect to a QGraphicsItem in this way, and it makes my QGraphicsEffect behave differently to the stock ones such as QGraphicsDropShadowEffect which don't require you to pass in the QGraphicsItem to their constructor.

Overall, you have to keep in mind that all QGraphicsEffect is giving you is a bitmap of your QGraphicsItem to draw on, which is obviously potentially useful, but also quite limited, and it might not be the best solution for your particular needs.