Updated Resource Filter UI

Here is a screenshot of the Resource Filter updated UI, as discussed in the previous post.

There is now support for OR, AND and NOT filter groups that allow the user to build arbitrarily complex logical propositions for filtering resources.

Resource Filter UI

An update to the previous post, I now included to the Resource Filter bug report a UI implementation.


The UI consist of a property page available when the user right clicks IContainer objects (basically folders, projects and linked resource folders), and allows him to add, edit, remove and reorder resource filters.

The interface allows the user to choose which resource filter type (currently either a regular expression or a String Matcher), the mode ("Include Only" or "Exclude All"), the target ("Files", "Folders" or "Files and Folders" the filter type arguments, and finally whether it is inheritable to the container's children or not.

Unless I'm mistaken, the result of the filter operation is always the same no matter the order of the filters (i.e. (A & B & !C == A & !C & B), so why allow the user to order them?

The reason why filters can be reordered is mainly for performance reason, since one can imagine filter types that are slower than others (one that check for file content types, for instance), and it is more efficient to put faster filters first so the slower filters have a smaller set to iterate through.

A side note, an interesting test is to put an "Exclude all" filter for files and folders matching "*" on a project itself: Will it cause the ".project" file to be filtered out as well? Thankfully, it doesn't, since the code path to retrieve the ".project" file is different than the regular local refresh. So the project still opens fine, although it is indeed an extreme case.

New Resource Filters for the Eclipse Core Resources

Our customers have to work with large source trees, where creating projects with those trees can be cumbersome.

Often the user wants to deal only with a subset of files in those hierarchies, but each file system object gets created an a resource in the resource tree and causes performance and scalability problems.

To address those limitations, we implemented the "Resource Filter" feature which we provided a patch to the core.resources plugin today (UI to come shortly).

With this feature, the Core Resources plugin now support adding resource filters onto IFolder objects that limits at a low level which file system objects are available to the resource tree so that large source trees can scale easily when filters are used.

How does it work:

• The .project file now contains tags that specify which filters exist on which folder.
  
• A folder can contain a list of Filters (from 0 to n)
  
• Filters can be optionally "inheritable", in which case they apply not only to the folder's children, but also to it's sub-folder children as well.
• Each filter has the following attributes:
  
  
  
  1. The project relative path of the folder it applies.
  
  
  2. The id of the filter type provider (ex: "org.eclipse.core.resources.regexFilterProvider").
  
  
  3. The arguments of the filter. This is filter dependent. (ex: ".*\.c")
  
  
  4. The filter type. This is a integer that can contain any of the following flags:
     
     
     
     • Filters can be set to include only files or folders matching a criteria, or exclude all file and folders matching a criteria. The two following flags choose between the two modes:
       
       public static final int INCLUDE_ONLY = 1;
       public static final int EXCLUDE_ALL = 2;
       
       Note that multiple filters on the same folder can be of a mix of "include_only" and "exclude_all" types.
       
     
     
     • Filters can be applicable to files or folders or both, the following flags indicate the preference:
       
       public static final int FILES = 4;
       public static final int FOLDERS= 8;
       
     
     
     • Filters can be inheritable to sub-folders, the following flag indicate that:
       
       public static final int INHERITABLE = 16;
       
  
  Note that all those flags are declared in the org.eclipse.core.resources.IFilter interface.
  

Something important to realize is that only real files that are located under a folder and appear in the workspace tree by a local refresh can be filtered out. Linked resource explicitly added under a folder still appear no matter what filters are put.

This is by design, since linked resources and resource filters are complementary to each other. Where linked resources allow the user to build the project structure by explicitly including files and folders, resource filters allows the user to build the project structure by explicitly excluding files and folders.

One practical thing to do, though, is to put resource filters on linked resource folders, which work perfectly.

Some interesting behavior:

• If the user creates a new file from the "File | New..." wizard that is actually filtered out by its parent resource filters, it will show up just after creation, but it will disappear from the resource tree when the user manually refreshes the view. I believe this is the proper behavior.


• Deleting folders that contain filtered out children work as expected (hidden children are deleted with their parent), but might cause the user to delete files that he's not aware are present on the file system. This should be addressed by the UI showing a warning in such case.


• Copying folders that have filters is interesting. While the filters get applied to the destination folder, only the visible objects are copied, which may or may not be what the user expects. Again, this issue should be resolved by having a UI giving the user the option (or at least the information) about what will happen


• Moving folders with filters, on the other hand, moves the whole content, including the hidden files invisible from the resource tree. This is because the move operation moves the underlying directory, and then copy the resource tree. The filters are moved as expected as well.

How to try it out:

1. Get the patch from:
   https://bugs.eclipse.org/bugs/show_bug.cgi?id=252996
   
   
2. Apply the patch to the org.eclipse.core.resources project.


3. Import the test project provided in this patch. The project contains two filters:
   
   • One filter on "folder" which "includes only" all files ending with ".c" (and is inheritable)
   • Another filter on "folder/sub_folder" that "excludes all" files matching "output.c".


4. Observe that the following objects are hidden from the resource tree:
   
   
   
   • "folder/foo.h" (the include_only *.c filter on "folder")
   
   
   • "folder/bar.h" (the include_only *.c filter on "folder")
   
   
   • "folder/sub_folder/file.h" (the inherited include_only *.c filter on "folder")
   
   
   • "folder/sub_folder/output.c" (the exclude_all "output.c" on "sub_folder")
   
   

Drag and Drop Improvements in Eclipse

One feature that IDE users are accustomed to is dragging files from Windows Explorer and dropping them on the application window to have them handled by that app.

With the 3.3 Platform, Eclipse added the "Editor area drag and drop" functionality, which aims at implementing this feature.

Unfortunately, there is still some limitations that need to be overcome to handle how users expect a native application to behave.

Handling files that open with external editors

The first thing that the user can notice about Eclipe's handling of dropped files is that for some file types dropped onto the workbench, external editors are launched to open the file type instead of Eclipse itself.

The most obvious example is when the user drags and drops a .bat file in the Eclipse workbench.

Instead of editing the content of the file in a text editor, as the user would expect, Eclipse actually launches the .bat file in a Windows command prompt, mimicking the same behavior as if the user double-clicked it in the navigator pane.

This is very inconsistent, since if the same file is dragged not from Windows Explorer but from an Eclipse project and dropped on the same editor, the .bat file is now properly open in an editor window instead of being launched in a command prompt.

This issue doesn't only affect .bat files, but all files that map to a system editor in Eclipse.

If the user wanted to simply have Windows to handle the .bat file and not Eclipse, he would have simply double-clicked it, and not took the effort to drop it on top of the Eclipse workbench.

Obviously, Eclipse doesn't do what user expect with those kind of files.

Lack of extensibility

Drag and dropping files in an Eclipse workbench demonstrate the general intent of the user to have Eclipse handle that file.

For some files, that may coincide with opening them in the editor, but with other files, Eclipse handling of the file may mean something very different.

For example, dropping an Eclipse project directory on the Eclipse workbench demonstrate that the user doesn't want to edit the folder, but import the project in the current workspace. The same is most likely with .project files as well.

Note that since the user can't currently double-click the .project file in Windows explorer and have Eclipse be open and handle the project, this is the only way he can have to conveniently import projects without having to go through the import wizard.

By dropping a .mcp or .vcproj file on the Eclipse workbench, the user demonstrate his intent of having those CodeWarrior and Visual Studio projects be imported in Eclipse, preferably by having a wizard come up.

Another thing that is typical for C/C++ IDE is dropping an executable on the IDE in order to start a debugging session for that executable.

In all those case, the expected Eclipse behavior by the user is different from what he would expect if he double-clicked the same files in the Eclipse navigator (in which case, he would expect to expand the project directory, to launch CodeWarrior or Visual Studio, and to run the executable respectively).

All those features are impossible to implement with the current Drag and Drop feature which is not extensible.

Our improvements to the Editor area Drag and Drop feature.

In order to overcome the limitations mentioned above, we made the following improvements in the Editor area drag and drop feature:

1. We extended the drop target for the general drag and drop workbench mechanism. Since the Eclipse workbench in general should handle file and folders dropped from Windows Explorer, the user can now use the top part of the Eclipse Workbench window to drop files.
   
   Note that we wanted to include the top most section of the workbench window, but we couldn't find a way to include it without including the whole shell at the same time.
2. We added the org.eclipse.ui.ide.dropHandler extension point to allow a flexible implementation and resolution of the handling of files and folders dropped on the Eclipse workbench (see more details below).
3. We provided default dropHandlers to handle files in a proper way (without resorting to external editors) and to handle project directories and .project files by importing the project in the current workspace.
4. We provided a conflict resolution mechanism that detects when two or more handlers declare supporting the same file and let the user choose between them in a convenient dialog.

Algorithm and Extensibility API

The general workbench Drop Handler algorithm relies solely on the dropHandler extensions to be able to handle dropped files and folders on the Eclipse workbench.

Each handler must have the following attributes:

• A user visible name
• An icon
• A handler class, implementing the IDropHandler interface (as described below)
• A unique string ID.
• A set of rules determining if a given file or folder is supported by the DropHandler.
  

The rules can be any combination of the following:

1. An inspector class, implementing the IDropHandlerInspector
2. Any number of typenames, which consist of the type of the object (file or directory) along with a regex pattern for the file name (such as "\.project").
3. Any number of file extensions, which consist of a pattern matching a file extension, such as "txt".
4. Any number of content types, which consist of the Eclipse content type IDs used in the Platform ContentTypeManager.
5. A lastInspector class implementing the IDropHandlerInspector as well.

When a file or folder is dropped on the Eclipse workbench, the workbench drop feature will iterate through all inspectors for all existing dropHandlers. If one inspector matches the file or folder, its handler class will be used to handle the file or folder and the matching process will end.

If no matching inspectors are found, all typenames will be tested, then all extensions, followed by all content types and finally all lastInspectors.

If two or more dropHandlers are found matching for the same rule level, the Conflict Resolution dialog will be displayed to the user to know which dropHandler to choose, unless a precedence already has been recorded by the user.

The following is an example of a Project Handler extension point:

<extension point="org.eclipse.ui.ide.dropHandler">
   <handler class="org.eclipse.ui.internal.ide.dnd.ProjectDropHandler"
       icon="icons/full/obj16/prj_obj.gif"
       id="org.eclipse.ui.ide.dnd.ProjectDropHandler"
       name="%DropHandlers.ProjectDropHandler">
       <inspector class="org.eclipse.ui.internal.ide.dnd.ProjectDropHandler">
       </inspector>
       <typeName name="\.project"
          type="file">
       </typeName>
   </handler>
</extension>

The IDropHandler interface is defined as follows:

package org.eclipse.ui.ide.dnd;

import org.eclipse.core.runtime.IPath;

/**
* Interface implemented by a dropSupport extension point to
* handle a file or folder dropped on the workbench.
*
*/
public interface IDropHandler {

  /** Handles a dropped file or folder on top of the workbench.
  * @param path  The full absolute path of the file or folder dropped.
  */
  public void handle(IPath path);
}

And the IDropHandlerInspector as follows:

package org.eclipse.ui.ide.dnd;

import org.eclipse.core.runtime.IPath;

/**
* Interface implemented by a dropSupport extension point to
* handle a file or folder dropped on the workbench.
*
*/
public interface IDropHandlerInspector {

  /**Returns true or false whether a handler support the type
  * of file or folder that was dropped on the workbench.
  * @param path  The full absolute path of the file or folder dropped.
  * @return true if the handler support handling the file or folder pointer by path, false otherwise.
  */
  public boolean handles(IPath path);
}

Conclusion

We hope that our improvements can be useful to other Eclipse users, and welcome your comments, questions and criticisms!

Our changes are freely available in the following Eclipse bug report, and we hope to be able to contribute the changes back to the Eclipse standard distribution:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=248462

That contains a patch with the implementation of our drag and drop feature improvement.

Improving Linked Resources in Ganymede - API Changes

Following up on the last post describing the user level changes of our linked resource improvements in Ganymede, I will now go over the underlying API changes.

A important requirement of the implementation of the linked resources improvement was to keep the resource API fully backward compatible: All code previously using the core.resources API should work just fine without any changes.

This is very critical, since so many plugins depend on the core.resources framework that breaking any contract would be unworkable.

Core Resources

The first change is the IResource.setLocation() methods as shown below:

class IResource […]
/**
* Sets the value of the link location for a linked resource.
*
* @param location
*            the new location of the target link resource
* @param updateFlags
*            bit-wise or of update flag constants ({@link #FORCE},
*            {@link #KEEP_HISTORY}, {@link #SHALLOW},
*            {@link #BACKGROUND_REFRESH} and {@link #REPLACE}).
* @param monitor
*            a progress monitor, or <code>null</code> if progress
*            reporting is not desired
* @exception CoreException
*                if isLinked() returns false.
*/
public void setLinkLocation(URI location, int updateFlags, IProgressMonitor monitor) throws CoreException;

/**
* Sets the value of the link location for a linked resource.
*
* @param location
*            the new location of the target link resource
* @param updateFlags
*            bit-wise or of update flag constants ({@link #FORCE},
*            {@link #KEEP_HISTORY}, {@link #SHALLOW},
*            {@link #BACKGROUND_REFRESH} and {@link #REPLACE}).
* @param monitor
*            a progress monitor, or <code>null</code> if progress
*            reporting is not desired
* @exception CoreException
*                if isLinked() returns false.
*/
public void setLinkLocation(IPath location, int updateFlags, IProgressMonitor monitor) throws CoreException;
}

As documented above, the IResource.setLocation() methods allow changing a linked resource location. The notifiers are then fired appropriatly, so that the UI is automatically updated when a location changes.

Second, since each project now has its own IPathVariableManager, its definition has been updated with the following methods:

class IProject […]

/**
* Returns the path variable manager for this project.
*
* @return the path variable manager
* @see IPathVariableManager
*/
public IPathVariableManager getPathVariableManager();

/**
* Returns a variable relative path equivalent to an absolute path for a
* file or folder in the file system, according to the variables defined in
* this project PathVariableManager. The file or folder need not to exist.
* <p>
*
* @param location
*            a path in the local file system
* @return the corresponding variable relative path, or <code>null</code>
*         if no such path is available
*/
public IPath getVariableRelativePathLocation(IPath location);

The method getVariableRelativePathLocation() will converts paths such as "C:\foo\bar.c" into "FOO\bar.c", granted than the path variable FOO exists already in the project PathVariableManager and it points to "C:\foo".

A more powerful method is the one available in the IPathVariableManager:

class IPathVariableManager […]

/** Convert an absolute path to path variable relative path.
*  For example, converts "C:/foo/bar.txt" into "FOO/bar.txt",
*  granted that the path variable "FOO" value is "C:/foo".
*
*  The "force" argument allows intermediate path variable to
*  be created if for a given path can be relative only to a parent
*  of an existing path variable.
*
*  For example, if the path "C:/other/file.txt" is to be converted
*  and no path variables point to "C:/" or "C:/other" but "FOO"
*  points to "C:/foo", an intermediate "OTHER" variable will be
*  created relative to "FOO" containing the value "${PARENT-1-FOO}"
*  so that the final path returned will be "OTHER/file.txt".
*
*  The argument "variableHint" can be used to specify to which
*  path variable the path should be made relative to.
*
* @param path  The absolute path to be converted
* @param force Set to true if intermediate path variables need to be created if the path is relative only to a parent of an existing path variable.
* @param variableHint The name of the variable to which the path should be relative to, or null for the nearest one.
* @return  The converted path
* @exception CoreException if this method fails. Reasons include:
* <ul>
* <li>The variable name is not valid</li>
*/
public IPath convertToRelative(IPath path, boolean force, String variableHint) throws CoreException;
}

Also, a new extension point "variableProviders" is available in the core.resources plugins that allows extending the default path variable list if need be.

Variable providers can be pecified as follows:

<extension

point="org.eclipse.core.resources.variableProviders">

<variableProvider

class="org.myPackage.myPlugin.MyPathVariable"

name="MY_PATH_VARIABLE">

</variableProvider>

</extension>

where "MyPathVariable" must inherit from the "IProjectVariableProvider", defined as follows:

public interface IProjectVariableProvider {

 /**
 * Returns a variable value
 *
 * @param variable
 *            The current variable name.
 * @param project
 *            The project that the variable is being resolved for.
 * @return the variable value.
 */
 public String getValue(String variable, IProject project);

 /**
 * If the variable supports extensions (specified as
 * "${VARNAME-EXTENSIONNAME}"), this method can return the list of possible
 * extensions, or null if none are supported.
 *
 * @param variable
 *            The current variable name.
 * @param project
 *            The project that the variable is being resolved for.
 * @return the possible variable extensions or null if none are supported.
 */
 public Object[] getExtensions(String variable, IProject project);
}

Groups

Groups are seen by clients as a broken folder linked resource. Since groups do not exist on the file system, calling IResource.getLocation() returns null, just as broken folder linked resource do.

All code that handle properly IResource.getLocation() returning null, as they should to support the possibility that a folder linked resource is not resolvable, will work transparently with groups as well.

The groups API changes are as follow:

class IFolder […]
/**
* Creates a new group resource as a member of this handle's parent
* resource. A group is not located anywhere in the file system, and can
* contain only linked files, linked folders and other groups.
* <p>
* This method changes resources; these changes will be reported in a
* subsequent resource change event, including an indication that the folder
* has been added to its parent.
* </p>
* <p>
* This method is long-running; progress and cancellation are provided by
* the given progress monitor.
* </p>
*
* @param updateFlags
*            bit-wise or of update flag constants (currently no flags are
*            relevant here)
* @param monitor
*            a progress monitor, or <code>null</code> if progress
*            reporting is not desired
* @exception CoreException
*                if this method fails. Reasons include:
*                <ul>
*                <li> This resource already exists in the workspace.</li>
*                <li> The workspace contains a resource of a different type
*                at the same path as this resource.</li>
*                <li> The parent of this resource does not exist.</li>
*                <li> The parent of this resource is not an open project</li>
*                <li> The name of this resource is not valid (according to
*                <code>IWorkspace.validateName</code>).</li>
*                <li> Resource changes are disallowed during certain types
*                of resource change event notification. See
*                <code>IResourceChangeEvent</code> for more details.</li>
*                <li>The team provider for the project which contains this
*                folder does not permit groups.</li>
*                <li>This folder's project contains a nature which does
*                not permit groups.</li>
*                </ul>
* @exception OperationCanceledException
*                if the operation is canceled. Cancelation can occur even
*                if no progress monitor is provided.
* @see IResource#isGroup()
*/
public void createGroup(int updateFlags, IProgressMonitor monitor) throws CoreException;

class IResource […]

/**
* Returns whether this resource is a group.
*
* @return <code>true</code> if this resource is a group, and
*         <code>false</code> otherwise
* @see IFolder#createGroup(int, IProgressMonitor)
*/
public boolean isGroup();

Improving Linked Resources in Ganymede

Coming from a C/C++ developer background, moving to the Eclipse CDT tools can sometimes be challenging.

C/C++ developers are used to work with standard IDE such as Microsoft Visual Studio and CodeWarrior, two products that have been designed from the start to be C/C++ IDEs.

One area that is specific with developing in C/C++ is the source file layout. C/C++ source files are divided in headers and sources files, are often shared between projects, and have no rigid layout in the file system.

Some teams group the C/C++ files by software component, some by framework, some by teams, and some by a bit of every division imaginable. C/C++ source trees end up being very complex, and it is of course out of the question to ask developers to structure their sources according to the way an IDE works, the IDE has to adapt itself to the way developers work!

Eclipse's solution for creating arbitrarily complex project structure is using linked resources.

Unfortunately, Eclipse's linked resources have the following limitations:

• Linked resources tend to contain absolute paths, unless the workspace's path variables are used.


• When path variables are used to avoid using absolute paths, the variable list is maintained in the workspace only, and if the user switches workspace, the project becomes broken.


• The linked resource's targets can't be changed by the user in the workbench!


• Linked resources can't be specified as being in a parent directory relative to a path variable (i.e. "PROJECT_LOCATION/../../dir/foo.c").


• Creating linked resources is very time-consuming, the user has to create each single file one by one through the new file wizards.

What we have done for our customers at Freescale is improve Eclipse's support for linked resources while maintaining full compatibility with standard Eclipse IDEs.

We can divide our improvements in 4 different sections:

Project Path Variable Manager

The first improvement consists of creating a IPathVariableManager implementation in each project, analogous to the one at the workspace level.

Then, the path variable list is extensible through an extension point, which allows default variable to be specified (such as PROJECT_LOCATION, ECLIPSE_HOME, etc...) and custom variable to be provided by 3rd party plugins.

For instance, a StarCore tool plugin may want to expose its SDK root as a path variable, so that each project can contain sources from the SDK, while having the source locations always pointing to the correct location.

A dialog has been added to handle drag and drop of files in the project, so the user can easily and quickly create a path variable relative project structure.

Special path variables are provided, such as "PARENT" which allows path variables to be specified as relative to the parent directory of another variable, effectively allowing arbitrarily complex project structure to be specified while keeping the project fully portable across computers, or "ENV" which allows linked resources to be relative to environment variables.

Since each project has its own PathVariableManager, special care had to be taken to be sure that copying linked resources that contain path variables between different projects causes the proper path variables to be copied as well.

Also, the user can now change the linked resource target in the file property page.

Groups

C/C++ developers are used to group source and header files differently than they are structured on the file system. In Visual Studio, the user can create filters in a solution, while with CodeWarrior, groups can be created. In Eclipse, the user has currently no choice but to create a set of empty directories, which is then cumbersome to maintain in a source repository and awkward to use.

For that purpose, we have added support for groups in Eclipse that the user can easily create anywhere in a project. A group works just like a folder, and is fully compatible with the existing core.resources API and all its clients, the only difference between a group and a folder is that:

• Groups are not created on the file system, they are instead recorded in the .project file.


• Only linked resources and groups can be created under a group, no real files or folder can, since a group is only a virtual container.

Internally, groups are implemented as folder linked resources.

Additional features

We have also provided a dialog that allows the user to list all the linked resources in a project and easily manage them.

The user can automatically convert absolute path linked resources to path variable relative, and fix broken linked resources.

We hope that our improvements can be useful to other Eclipse users, and welcome your comments, questions and criticisms!

Our changes are freely available in the following Eclipse bug report, and we hope to be able to contribute the changes back to the Eclipse standard distribution:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=229633