Sifter Blog

Sifter Development Process and Tools

2013-04-29T09:25:12-05:00

1. Ideas and Discussion

Almost every idea begins as a result of support requests. With Help Scout, we also tag and archive all of our old support requests. That way, once we start development on a particular feature, we can go back through old support requests to get insight into the underlying customer problems that originally sparked the idea.

With almost any interesting idea, we’ll create an issue in our “Ideas” project regardless of whether we think it really has a chance. The issue will generally live in there for some time while we discuss it and work on other features. In the ideas project, we explicitly don’t use milestones. That lack of milestones combined with the fact that it’s separate from our main development project, we’re able to stay more focused on today’s work rather than what we might be working on in the future. It also helps us treat the ideas project as more of a blue sky project without any obligations.

Figure 1 Support requests in Help Scout drive the majority of our ideas and ultimately new development.

Depending on the idea, we’ll often create a branch at this point and just start prototyping and playing with the idea to see how we like it. Usually this just helps us fill out the idea and we add thoughts back to the issues. Every now and then, though, the idea will turn into an unstoppable force and we’ll just build it.

2. Development

As we complete work on one thing, we start thinking about what we could work on next. We usually have a good idea of what that will be, and it’s almost always already heavily documented in our ideas project. We’ll generally take that issue, create a milestone in our main “Sifter” project, and begin logging issues against that milestone based on the discussion in the original issue.

Once we’re satisfied with the work laid out in the milestone, we’ll create a Git branch with the same name as the milestone for handling the development work without affecting our master branch. Once we start writing code, Travis and Code Climate are constantly monitoring our work to make sure that we’re writing high quality code and not introducing any new bugs. If anything goes wrong, they’ll send us an email and post a message to Campfire.

Figure 2 We use Sifter to organize and drive the majority of our development. Once code is pushed to GitHub, Code Climate and Travis step in and help make sure we didn’t write any bad code.

Once the branch and milestone are up and running, we simply start iterating. As we’re writing new code and resolving/closing issues, we’ll occasionally release the work in progress to our staging environment so that other team members can see what’s happening and share their feedback without having to do the work and run the branch locally.

Figure 3 Milestones in Sifter help us gauge how much work is left to do on any given feature.

3. Review, Internal Testing, and Dark Launch

As this process continues we eventually get to a point where virtually all of the issues are closed and only a few superficial items are left. Then the lead on that milestone will open a pull request on GitHub.

Figure 4 GitHub’s pull requests are great in that they clearly communicate whether the code can be merged cleanly as well as whether it has pass all of our tests via Travis.

This pull request signifies that while the code may not be 100% ready to ship, it won’t be changing drastically, and other team members can start reviewing the code and sharing their thoughts. This helps us catch any oversights in logic or performance and gives us all a chance to familiarize ourselves with everyone else’s code.

The pull request also signifies the point where we know we can start exhaustively testing the new code on staging and logging any final bugs in Sifter. Sometimes we won’t find anything, and other times, we’ll uncover a handful of bugs. Either way, we’ll get to work closing out the last few issues, and once all of the kinks are worked out, the pull request is ready to go.

4. Merging and Releasing

As soon as all of the issues are closed in Sifter, and Travis green lights the pull request, we’ll do one last release to staging and run through the relevant functionality just to make sure no new bugs slipped in at the last second.

After that, we’ll go ahead and merge the new branch into master via the pull request and push it to production with a final production smoke test to make sure that everything is running smoothly. We borrowed Nathaniel Talbott’s post-deploy smoke tests to add simple automated smoke tests to our release process, but we almost always do manual spot checks as well. It’s rare that we find anything, but when we do, we’re always glad that we were able to catch it sooner.

Figure 5 While not every release merits going through step 2, it’s included here for completeness. This process helps minimize the chances of any bad code sneaking into production.

5. Monitoring and Analytics

Of course, once new code is in production, the development process isn’t over. No matter how careful a team is, some bugs won’t show up until your code is exposed to real world usage. For this, we rely heavily on New Relic. If a new release has an usually high error rate or slower performance, we’ll be able to quickly identify the problem and take steps to address it.

Simplifying and Automating the Development Process

We didn’t implement this process overnight. It has slowly evolved along with the tools that we use and our team size. Outside of me, nobody else works on Sifter full-time, so our communication is incredibly asynchronous. Even Sifter’s role in this process has evolved over time. For instance, separating our active development from our backlog of ideas is a recent development in the last year or so. Similarly, our usage of GitHub pull requests has evolved over time as more people have contributed to Sifter.

There’s no doubt that having a consistent process helps, but we’ve found that it’s best to let processes evolve organically as the team grows or as your technology and infrastructure evolve.

How We Use Projects in Sifter

2013-04-26T09:05:12-05:00

These days, we have four active projects. One each for Sifter, Marketing, Ideas, and the book. The Sifter project is self explanatory. This is where we track any work that needs to be done related to the application or the infrastructure where it lives. Marketing lets us manage any advertising, blogging, emails, or updates to our marketing web site. Ideas is much more open-ended, and the book project is for handling everything from typos to how we might offer additional formats.

Figure 1 Internally, we use a few different projects to help us stay organized and keep different areas of work separate.

Sifter

The Sifter project is primarily organized by milestones. These almost always correlate to a branch or pull request on GitHub. However, we also make heavy use of the “Unplanned” milestone. Any minor bug fixes or other tweaks that need to happen but don’t justify a dedicated milestone get filed without a milestone. For us, “Unplanned” doesn’t mean “No Plans”, it simply means that it is work that needs to happen but isn’t part of the plan. Another way to think of these issues is “Hot Fixes”. While milestone-centric updates are less frequent, these smaller unplanned updates happen almost daily.

Beyond that, we try to keep 3-5 active milestones. In case, this is primarily a factor of team size and availability. Since I’m the only person full-time on Sifter, and I’m not doing development full-time, we really don’t have any full-time developers. We’ll generally have an active milestone for each person’s main focus. Then, once that milestone reaches a point where we can actively discuss its direction, the person leading the charge on that effort will open a pull request on GitHub. From there we’ll do some testing, review the code, and, if necessary, log new issues or questions against the branch.

Figure 2 While Sifter helps us stay on top of our bugs and issues, GitHub’s pull requests help us get the code into production.

Marketing

The marketing project is less exciting, but it’s a great example of how well Sifter works for handling much more than just bugs or issues. One of the most useful ways that the marketing project helps is by keeping tracking of likely topics to blog about. Most of the inspiration for blog post topics is a direct result of answering support emails, but keeping a backlog of potential topics helps us recognize when we’ve heard enough questions to address the topic. If that topic seems like a good one, we add comments as we evolve the idea. Then, when we’re ready to blog about the topic, we have a collection of thoughts around what to write about. Similarly, any changes to the marketing web site go through the marketing project so that they don’t distract from core development.

Ideas

We originally filed all of our “someday” ideas inside of the Sifter project. However, we’ve found that it’s much more productive to keep them out of the way. If an issue is in the Sifter project, we fully expect to take action on it at some point in the near future. Otherwise, it doesn’t belong there. The main project is for action, not talk. The ideas project gives us a place for the talk and is sub-divided by categories like API, Billing, Notifications, Search, etc. So whenever we decide to work on a specific area, we can also hop into the ideas project and filter the ideas to see if there are any relevant enhancements we could knock out at the same time. Naturally, since the ideas project is more of a sandbox, we don’t use milestones for it at all.

With the ideas project, though, we can toss any half-baked idea into the project with reckless abandon. Think of the ideas project as a safe place for good and terrible ideas alike. If we can easily log an idea and get feedback from others, we can slowly build both a detailed backlog as well as a collection of knowledge around our decisions. Once we make the decision, we can close the issue. Then if it comes up again the future, we just reopen the discussion. If we decide to do it, we can either move the issue into the Sifter project or, if it’s large enough in scope, create a milestone for it in the Sifter project.

Figure 3 With our ideas project, we can maintain active discussions or even revisit old decisions without bogging down our primary project.

For instance, we might have a discussion about Markdown support where we’ll talk about the pros and cons until we reach a point where we’re comfortable with a decision. Then, if the topic comes up again a month later, we have a clear record explaining our earlier decision. We can then easily evaluate whether our original concerns are still valid. By documenting our thought process, we know we can always go back and question our original assumptions.

Book

Writing a book is an interesting project. Most of the issue tracking revolves around editing, but there are many other components that need to be tracked on an ongoing basis. Books have bugs, todos and issues just like software. Cover design. Payment processing. Formatting. Fonts. The web site. Announcement email. Typesetting. Having a central place to track and discuss these items makes it easy to stay on track. And by keeping it in the same location as our software development projects I was able to more easily juggle multiple projects without missing anything.

Issue Tracking as Project Management

Sifter is a great bug and issue tracker, but can also serve as a great project management tool. It’s explicitly designed to be abstract enough to track much more than bugs and issues. We believe that everything needs an explicit resolution. Sometimes that resolution may simply be “tabled” or “out of scope”, but by saving those resolutions for future reference, we’re able to maintain a starting point for any future discussions without rehashing the same old stuff every time. Open-ended threads about a topic are alright, but the value of always thinking in terms of closure, resolutions, and accountability help maintain forward progress and keep everybody on the same page.

Bug and Issue Tracker Workflow

2013-04-25T21:49:43-05:00

Figure 1 Sifter uses a carefully considered set of statuses to provide a simple and flexible workflow or managing issues.

One of the most important parts of keeping bug and issues organized is making sure that everyone is on the same page with regard to statuses and uses them correctly. The most important thing to remember is that issues don’t have to be resolved before they can be closed, and in our case, we frequently skip resolved and go straight to closed if the situation warrants it. There are situations, however, where stopping at resolved is key.

Varying Degrees of Doneness

Before we go into how we do things, let’s talk about the benefits of having two layers of donneness in an issue tracker. With software, the cost of fixing a bug before the software ships is always less than the cost of fixing it after it has shipped. Even with today’s teams that are able to constantly ship new versions all day every day, nobody wants their customers to deal with bugs. So you want to be incredibly sure that when you fix something, you fix it correctly. Moreover, it’s simply more expensive to do something twice than it is do it once.

If you want to make sure that it gets done right the first time, you’ll invariably want a review layer within your workflow. With bugs, “Resolved” represents “Fixed” or “Ready for Review”. In general, a developer will fix a bug and mark it as resolved. Then, the original opener will be able to review the fix and make sure that the developer fixed the problem as expected. If everything looks right, the opener then closes the issue and moves on to the next one. This helps prevent simple miscommunication or other errors from turning into shipping bugs.

Resolved

With our team, “Resolved” generally means that the fix has at least been committed to source control. Any potentially unusual situations with resolved issues are addressed with a detailed explanation, or resolution, in the comments. Don’t underestimate the importance of writing a clear and detailed resolution. It’s easy to be in a hurry and want to simply close the issue, but taking a few extra seconds to type a detailed resolution can go a long ways to preventing future miscommunication.

Depending on whether it’s a one-off fix or part of a larger release, the fix may even be deployed to staging for easy testing. With small one-off fixes, it’s unlikely that we’d deploy to staging, but when we’re working with larger milestones, we’ll regularly release the most current version of the relevant branch to staging. For us, staging is a sort of shared public space for easy previewing and testing of significant releases for the entire team.

We also use staging as the final step of validating significant infrastructure changes. Our staging environment almost exactly mirrors our production environment, load balancer and all. For instance, if a bug is significant or relies on functionality that works slightly different in development than it does in production, then we never want to close it without reviewing it in staging. So if we make a change to our search indexing or cron jobs, testing in development isn’t enough. In these cases, we’ll mark the issue as resolved. Then, when we’re ready, usually when all other issues in the milestone are closed, we’ll release to staging and run through final checks on all of the resolved issues before closing them out.

There are several types of issues where it makes sense for an issue to stop at “Resolved” before it’s officially closed. This usually depends on the type of issue as well as the people involved. You can classify most of these scenarios as approval or review.

Approval. In these scenarios, a decision has been made or a feature implemented where someone needs to approve it. For instance, if a couple of us decide on a course of action that will lead to significant server modifications, we’d want to mark an issue as resolved and wait for our system adminstrator to chime in with his final approval.
Review. In these scenarios, it’s all about having a second set of eyes on things. If the issue opener and assignee are diferent, it’s easy for the developer to incorrectly believe that a bug has been fixed. By having the original opener review the fix or by retesting the issue in our staging environment, we’re able to have a higher degree of confidence that it’s truly fixed.

Figure 2 When an issue is resolved, we automatically adjust the interface to help visualize the forward progress of closing an issue as well as the nature of taking a step back if you reopen an issue.

Just as it looks in the interface, “Resolved” is a liminal zone where the problem is solved but needs verification or approval to be 100% complete. It’s perfectly valid to reopen the issue and bring the discussion back to life. In our ideas project, we’ll often promote issues to “Resolved” and leave them there until we’re ready to proceed further. We aren’t quite ready to close the issues, but we do want to acknowledge that we’ve tentatively made a decision.

Closed

Once an issue is closed, it’s ready to go. “Closed” doesn’t mean “Live.” It simply means “Closed.” That is, it’s either fixed and has been reviewed or it’s something that doesn’t currently need any further attention. You could say it’s basically like saying, “Nothing more to see here. Move along.” When all issues in a given milestone are closed, it’s a safe bet that the associated branch should be ready for release to production.

Like resolving an issue, there are two subtle variants behind the meaning of “Closed”. Either it’s complete and ready to go live, or it’s something that either isn’t going to happen or is no longer relevant. In either situation, no further attention is required at this time. “Closed” can even work as a sort of “On Hold” status if your team is comfortable using it that way. Remember, it’s dangerous to explicitly have an “On Hold” status, and Sifter provides several alternatives to achieve the same goal.

Complete. In these scenarios, the issue is both resolved and reviewed. It’s ready to go, and nothing more needs to be done.
Back burner. Closed doesn’t have to imply “fixed.” You can think of closing an issue as a simple way to get rid of an issue if you’re unlikely to do anything about it in the near future.

Figure 3 When an issue is closed, we automatically limit the options and help visually communicate the nature of taking a step backwards when reopening an issue.

It’s easy to forget, but along with closing issues, anyone can always reopen issues. Issues never die, they’re simply moved off of your todo list. They can always be brought back to life with a simple comment. Internally, this happens all of the time as we revisit old ideas or think of new ways of looking at ideas we previously believed weren’t right for Sifter.

Nothing is Absolute

It’s critical that multiple layers of doneness are available in a bug tracker, but those extra layers are only a suggestion for when it’s relevant to the given issue. Don’t be afraid to close issues or skip the resolution step if there’s no review necessary. It’s structure, but it’s designed to be optional structure. There when you need it. Gone when you don’t.

Sifter Notifications & Google's Spam Filter

2013-04-25T21:50:00-05:00

What’s happening?

While most emails from Sifter are being delivered without problems, some notifications, usually shorter emails that only include information about updated status, priority, assignee, etc. are ending up in spam folders. While we haven’t been able to determine an exact cause, we are confident that only issue and comment notifications are affected.

We’ve been working with our transactional email provider, Postmark to extensively test our deliverability and make some extra upgrades to be sure that we’re in great shape as far as sending reputation goes. Postmark is also great about handling spam, and despite sending millions of emails, we’ve never had a notification reported as spam. We’re still working to solve the problem, but at this point, we need to enlist customer assistance to let Google know about the false positives.

How do we fix it?

There are a couple of key steps to help straighten this out and prevent Sifter notifications from accidentally ending up in your spam folder.

If you use Google for email, check your spam folder. If there are notifications from Sifter, please flag them as not spam so that Google is aware of the false positives. It’s important to not that if you use multiple Google-backed email addresses and forward the emails from one to the other, your Sifter notifications may be caught in the spam filter of the first email address before being forwarded along.
Add to your Google mail address book.
Add a filter that ensures that emails from are never sent to spam.

We believe that things are getting better as we’re receiving fewer reports of missing notifications, but we also want to be sure that as few people as possible are affected. Of course, if you have any questions or concerns, please don’t hesitate to get in touch.

Notifying Additional People

2012-12-12T09:36:10-06:00

Now, when you create an issue or add a comment, you can mention people by their username, and we’ll send them a one-time notification. This gives you the ability to selectively pull people into a conversation when you need to without adding clutter to the interface. The auto-suggest feature will work based on username as well as first or last name so that you don’t have to memorize usernames.

Figure 1 Autosuggest will help you find usernames so that you don’t have to remember them all. Then, we’ll send a notification to anyone that you mention.

How does it work?

When you mention someone, they are not automatically subscribed to the thread. Instead, we’ll send them a special one-time notification that you mentioned them so they can add a comment or follow the issue. This way, issue notifications are opt-in. So, if I mistakenly mention the wrong person, and they feel like they don’t have anything to contribute, the burden isn’t on them to unfollow the issue.

It will also work if you mention people’s username when replying to Sifter notifications via email. Naturally, you won’t get the benefit of the auto-suggest feature from within your email client, but the feature will work other than that.

Sifter & Markup

2013-04-25T21:50:25-05:00

It’s taken a lot of careful consideration and feedback over the years to reverse course on this, but I now fully believe that the benefits of escaping HTML outweigh the drawbacks. The goal with this change isn’t to enable huge blocks of code but rather add flexibility for developers when discussing bits of code.

For the time being, we still recommend using Gist or Snipt if you need to discuss large blocks of code, but the ability to discuss some basic markup without leaving Sifter should help make communication a bit easier.

One other benefit is that replying to and creating issues via email should be much more reliable. Previously, email signatures had a tendency to include bits of markup that would trigger our markup filtering and reject some emails. With the new policy of accepting and escaping HTML, it should be much more forgiving. However, you may have a lot of additional email signature cruft.

This change won’t be live right away, but it will happen in the very near future. I wanted to go ahead and start giving everyone a heads up about the change. This change will also be retroactive for all issues and comments. If you have any questions or concerns, just drop us a line.

File Uploading 2.0

2013-04-25T21:50:36-05:00

Figure 1 Use a modern browser and drag and drop your way to file uploading bliss.

Figure 2 If you’re uploading images, we’ll even show you a thumbnail preview so you can make sure you added the correct files.

Drag & Drop

First and foremost, if you’re using a modern browser (Latest versions of Safari, Chrome, Firefox, and Opera as well as IE10) you can now drag and and drop files from your computer directly into Sifter. You no longer have to browse your computer for files. Of course, if you prefer to browse, you can. In fact, browsing has been improved as well.

Selecting Multiple Files

If you like browsing files and your browser supports multiple file selection, you can now select multiple files at one time.

Reordering

Once you’ve selected and uploaded files, you can now reorder them via drag and drop, and we’ll remember that order when the files are created.

Thumbnail Previews

If you’re uploading images, we now automatically generate a small preview thumbnail so that you can be sure you are uploading the correct files.

Speed & Performance

We no longer submit the files with the form. Instead, we upload and save the files as soon as you add them. This way, you don’t have to wait around while we process and upload your files.

Maintaining Uploads on Errors

It used to be that if there was an error submitting your issue, you’d need to reselect your files because we failed to maintain them on errors. That was really lame, but given the low incidence of errors due to the rather flexible validation, we let it slide. No more. If you encounter an error, your files will still be there.

Error Handling

Before these updates, if we encountered a problem with a corrupt or otherwise unreadable file, we just sent a generic “we couldn’t handle that file” error with a little bit of explanation. If you were uploading multiple files, you had no way of knowing which file was causing the problem. Fortunately, with the new system, if we encounter an error with a particular file, we’ll let you know exactly which file caused the problem.

File Size Limits

Prior to these updates, we kept the file size limits low because the process of uploading and processing 5 files could mean really slow issue creation times. We recently increased the file size limits to 30mb, and we’ll be keeping an eye on it to see if it makes sense to go even a little higher than that.

Browser Compatibility

If you’re still using IE8 or IE9, which for Sifter is very few people at this point, now would be a great time to upgrade that browser or switch to Chrome, Firefox, or Opera. File uploading will still work, and you’ll enjoy some of these benefits, but you won’t be able to take advantage of the drag and drop uploading.

Follow and Unfollow

2012-11-01T09:55:25-05:00

Figure 1 You no longer need to comment to add yourself to the list of notifications for an issue. Just click.

We believe that you shouldn’t have to think about notifications. They should just work. As such, Sifter has always had incredibly simple notification rules. The simplicity is great in some ways, but in other ways it was too restrictive. We’ve spent an incredible amount of time thinking about how to address the shortcomings without dramatically complicate things, and add the follow/unfollow button is the first step.

Figure 2 Done participating in a particular thread and no longer want notifications for it? All it takes is a click.

Sifter explicitly doesn’t allow you to send mass notifications to the entire team because we feel that increases the noise without increasing the quality of the notifications. (Although, we are working on adding @mentions so you can ping relevant team members when commenting.) By preventing people from unsubscribing, we were also contributing to the noise. Thankfully, you can now step out of a conversation when your expertise is no longer necessary.

This change should have a meaningful impact on the quality and relevance of Sifter notifications. We know it’s only one small step, but there’s much more on the way.

Twitter & Customer Support

2012-11-01T07:51:29-05:00

There are a variety of reasons that Twitter impedes our ability to help. We certainly try our best, but your best bet will always be contact us directly using our support request form. So, why is Twitter such a bad medium for customer support?

It’s Unreliable

This is the single biggest problem. We have notifications setup through both Twitter and iPhone apps. Unfortunately, it’s not uncommon for us to not receive notifications for a day or two, and sometimes, the notifications may not show up at all. We strive to respond to all emails during business hours in about 30 minutes. We’re not perfect, but we usually respond in that same amount of time outside of business hours as well. When we don’t even know someone has asked for help, it’s difficult for us to provide the level of support that we’d like.

It doesn’t tell us who you are.

Many bugs or support requests require that we look at your account information. With Twitter, while we might be able to look your account up by your name, that’s not the most reliable method. Many people don’t include their real name or their full name on their profile, and even then, there’s no guarantee that you’ll be the only person in the system. If you use email, or even better, our support request form, it’s much easier for us to look up your account so we have the best information to provide an accurate answer.

It prevents you from adding details.

One of the most important part of support requests is the request itself. Unfortunately, 140 characters is rarely enough to truly explain your question or problem. As a result, we often have to ask for more information or just take a guess about what you mean. The former slows the process down, and the latter could have us troubleshooting the wrong thing entirely. In either case, providing a thorough support request can help us help you faster.

It prevents us from providing useful responses.

Some questions can be asked in 140 characters, but it’s rare that thorough answers can fit in the same space. If we need to add screenshots or other attachments, the task is further complicated. We want to provide the best answers possible, but Twitter simply won’t let us do that. Of course, there are tools that enable us to provide longer responses, but that doesn’t solve the other problems.

Summary

We want to provide top-notch support, but unfortunately, Twitter complicates that. We want Twitter to be a good channel for communicating with folks, but if you need support or want a thorough answer, we recommend coming straight to us. You can use the support request form or simply email support@sifterapp.com. We’ll still do our best to help via Twitter, but we can be more helpful via email. Thanks for understanding!

Tagging is Messy

2013-04-25T21:51:05-05:00

We understand that some teams like to use tagging as a means to categorize issues, but they can be categorized just as easily with and much more safely with, well, categories. It’s totally possible to create as many categories as you want to help keep issues organized. We wouldn’t advise that, but you can.

Typos are Dangerous

If you have a typo when tagging a photo, there’s probably not a real risk. However, if you accidentally type “securty” instead of “security” when tagging an issue, it’s a very different story. When you go back to find all of the “security” problems and make sure that they get fixed, it’s likely that the “securty” tag will slip through the cracks, and you may end up overlooking an important issue. There are some ways around this, but they defeat the purpose of having an open-ended tagging structure.

Using the Right Tags is Difficult

When a category is created, everyone understands its purpose and can easily decide which issues fall into that category. With tags the story is a bit different. What makes sense as a tag to me, might not make sense as a tag to you and when it comes time for you to submit an issue you might decide to use a different set of tags. For example, one person may classify an issue as “security” while another might say “login” and yet another might say “session”. After a bit of time the system has a whole set of tags that contain their own individual issues, which defeats the purpose of having them. The problems become similar to the problems with typos, and problems can unfortunately slip through the cracks.

Organizing Tags is Counter-productive

The more tags that build up, the harder it becomes to keep them organized. Having to spend time organizing issues and their tags is not an ideal way to get the issues resolved. Having someone worry about what tag they should use for an issue might also make them a bit hesitant to submit an issue or make them hesitate long enough to get distracted and we understand how difficult it can be to get someone to take the time to report issues as it is.

Summary

We want the issue tracking flow to be as simple as possible. Report, resolve, and review. The more steps that get added in between or the more thought required, the longer it takes to resolve issues and that means less time crafting new features and products. In theory, tagging sounds great, but for the time being we feel like it’s not the right feature for Sifter.

Simplifying Accountability in Sifter

2012-08-26T09:35:11-05:00

Figure 1 The original status bar wasn’t bad, but it was cramped and all attributes were treated the same with no room for differentiation.

We prefer to avoid overly complex permissions in Sifter. It can create confusion or lead to mistakes where people are accidentally granted the wrong permissions. Instead, we rely on accountability so that individuals can be held accountable and understand they they are accountable for their actions.

With a recent release, we made a very small but important change to emphasize this focus on accountability. On the issue detail page, all of the issue’s primary attributes are displayed in the right-hand column. However, in addition to these attributes, we now display the name of the person responsible for setting that attribute.

To make it even more useful, you can click on the individual’s name, and we’ll highlight and jump you to the comment through which they made the change. This way, you can quickly review the reasoning to understand why they made the change.

Figure 2 When you click a user’s name next to an attribute, you can quickly and easily view the comment that they made in association with the change.

The entire philosophy behind Sifter is about openness and inclusion. We want non-technical people to feel like welcome participants in the issue tracking process. That openness and inclusion ultimately relies upon the assumption that if people have access to a project, they’re all on the same team and working towards the same goals. Instead of preventing people from doing or accessing specific capabilities, we believe that it’s easier and simpler if we enable people to hold each other accountable.

Tips for Effectively Reporting Bugs and Issues

2012-08-23T09:35:11-05:00

Avoiding putting in the effort up front may help file the issue faster, but it’s likely the developer will still need to request additional information. So, skimping on details is generally only delaying the inevitable. It’s best to put in the time up front to file a good bug report, and everyone will be more efficient.

Ultimately, every bug is different, and all of this information may not be required every time, but this should help provide some context to writing the best possible bug reports.

Report Them

It may seem silly, but the single most important part of bug tracking to make sure they get into the system. Even the smallest or most obvious bugs are often times rare or otherwise easy to miss for a variety of reasons. The important thing is that if you see a bug, file it. Don’t worry about whether it’s a duplicate. It’s important to get it into the system. In a lot of cases, a particular bug may be difficult to reproduce, but if several instances are reported, it’s easier for the developer to find a common thread and solve the problem.

Only One Problem Per Issue, Please

Each issue ultimately needs to be resolved and closed. If a single issue lists multiple bugs, then there’s no way to close one of the bugs if the other two are still open. As a result, the issues need to be split into several issues so that they can be closed individually. This creates room for mistakes, confusion, or other misunderstandings if any information is lost when splitting the issues. Also, different team members are often responsible for different areas of an application. So, by dividing it into separate issues, you’re also enabling each issue to be giving a unique assignee and priority.

For instance, a bad issue would look like…

The login form doesn’t work. The forgot password form is saying my user account doesn’t exist. The link is broken to return to the login form.

While a good issue would split each of those into three separate reports so that they can be handled individually. Then one person could handle the broken link issue quickly while a different developer can look into the login form issues.

Use Descriptive Subjects

When viewing a list of issues, the subject is the most important identifying piece of information. Think of it like the issue’s name. For instance, you wouldn’t refer to your best friend as “the person”. That could be anyone. You’d want to use your friend’s name. The same goes with issues. “Broken Login” isn’t detailed. “Can’t login due to missing user account” is much better. The more descriptive and accurate that it is, the easier it is to both locate a specific issue and quickly understand what it’s about. That helps make the team more efficient and means quicker resolutions on issues.

Write Well

The majority of communication around a bug or issue is text. So, it would make sense that writing is one of the most important pieces of filing a bug report. It’s always helpful to use complete sentences and make an effort to clearly identify the problem. Any lack of clarity in the communication can lead to misunderstanding or other errors. That can lead to requests for more information which take time and can drastically slow down the resolution process.

Focus on Facts

It’s often tempting to suggest or speculate what you think the problem may be, and at times, this can be helpful. However, more often than it not, it can be a distraction and actually make it more difficult for the developer to get to the bottom of things because it may send them down the wrong path. So it’s generally better to focus on objectively reporting facts about the bug rather than trying to guess what the source of the problem really is.

The Bug Reporting Format

Beyond writing well, there’s a simple format that you can rely upon for writing great bug reports. Remember, developers aren’t mind readers, and sometimes it helps to really spell things out.

What did you expect to happen? This is important. Some bugs are really just subjective usability issues. Reporting an issue without clarifying your expectations may lead to a “working as designed” response. However, with the added clarification of your expectations, it’s easier to understand the anticipated behavior and determine whether it’s truly a bug or a usability issue that needs to be reviewed.
What actually happened? Did you see any errors? Do you have a URL or screenshot for the page to which where you were ultimately taken? Sometimes, when an issue is diffuclt to reproduce, seeing a screenshot or knowing an error message can be the key to solving the problem.

For instance, saying “can’t log in” or “login is broken” doesn’t help, and the developer would invariably need to request more information. Those are very open-ended and leave a lot of room for possible problems. However, adding a little more detail could really help. Here’s an example of a better bug report. It’s still brief, but it provides just enough additional information for the developer to dig in.

I’m at (URL) and I’m trying to log in using my username, (username). However, the system is saying that the username cannot be found and taking me to (URL).

Beyond that, anytime you can provide screenshots, URLs, error messages, or any additional information it can work wonders helping the developer get to the bottom of things. Even the smallest detail can be useful when tracking down a difficult to reproduce bug.

Document Steps to Reproduce

It’s always important to expect that the people on the other end will not be able to reproduce the problem. There are too many potential variables involved. If a bug can’t be reproduced, it can be incredibly difficult, if not impossible, to fix. By providing the context of what you were doing when it happened as well as the minimal steps you followed to createt he issue, you’re ensuring that the issue can be reproduced without much effort.

First, always document the problem as best you can as soon as you encounter it. Then, try to reproduce the problem. Once you’ve reproduced the problem, document the minimum sequence of events that’s necessary to reproduce it. If you can, make sure to include screenshots or video.

Provide Additional Relevant Information

Finally, it’s helpful to understand the types of information that can be extremely valuable to developers when reporting an issue. These bits of information aren’t universally helpful, but they make a difference often enough that they’re worth mentioning.

URLs

If it’s a web-based application, providing the relevant URLs is amazingly powerful. URLs often contain bits of data that can help the developer track an issue down to a very low level. With URLs, it often also helps not only to provide the URL where the error occurred, but also the URL for the page you were on just prior to the error occurring. The more URLs, the merrier.

Error Messages

Anytime there’s a bug, and an error message of any kind is displayed to you, the contents of that error message are very important. Wether it’s a user-friendly error message or a system-generated error message that looks like gibberish, those error messages go a long ways in helping a developer understand what happened.

Operating System, Web Browser, Device, etc.

One of the most common sources of bugs is the variation in different platforms. Some bugs might only occur in Windows while others only happen on Macs. The same goes for Android and iOS. Then something might be broken in Firefox but work perfectly in Chrome. So when reporting an issue, it’s generally helpful to include your browser and operating system information.

Summary

A bug tracker is ultimately only as good as the bugs that get filed. By taking some extra time to thoroughly document and report bugs up front, you can save time and ensure that the bugs are resolved more quickly and with less back and forth. It may take a few extra moments, but it’s almost always worth the up front investment.

The Challenges with Custom Statuses

2012-11-25T16:35:44-06:00

We appreciate the potential benefits of custom statuses but feel that for the majority of teams, those benefits are outweighed by the advantages of simplicity. Of course, the world isn’t always as simple as open, resolved or closed. So Sifter has a few other features designed specifically to compliment these fixes statuses. In designing Sifter, we have always had Conway’s law in the back of our mind.

…organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.

It’s as if there’s an inherent gravitational pull towards complexity. It’s attractive, and it tries to sneak in at every opportunity. Unfortunately, complexity doesn’t just end with software. The fallout from complexity can be far-reaching and go far beyond the confines of software. Fighting complexity is an ongoing struggle. A little here, a little there, and before you know it, everything is a mess.

Naturally, there are teams that truly need this level of complexity in their workflows. However, our experience has been that for the majority of small teams out there, that complexity does more harm than good. We certainly don’t expect NASA to start using Sifter any time soon, but we feel that simple is best for most of us.

Figure 1 There are some teams doing incredibly complex things that may need a fully qualified workflow like above, but our experience has been that the majority of teams out there can condense most of those steps into a much simpler and easier to manage workflow.

The Complexity Monster

Custom statuses create complexity, and complexity doesn’t add up; it multiplies. Custom statuses require configuration. Configuration requires some preliminary thought about the different states. Too many different states, and people get confused about which state is right at which time. That leads to more thinking or asking about which status is right. Too many statuses can also lead to overclassification and organization fragmentation. When that happens, issues slip through the cracks. All of these things compound and create a variety of problems.

A Distinction without a Difference

If you look at the diagram, you’ll see that all Sifter has really done is do away with the multiple levels of distinction within each status. For instance, “accepted” or “assigned” are implicit. We just assume that all issues are accepted. If they aren’t accepted, then they need to be resolved and closed as such. Similarly, if an issue has an assignee, it’s been assigned. There’s no need to double up on that classification. We’ve shared our thoughts on why “on hold” and “pending” can create problems of their own. And finally, something is implicitly “in progress” if it’s open and assigned. Adding a status to capture that information is not only redundant, but tedious.

Track All the Things

If a set of statuses is overly specific, then more statuses can remove flexibility. “Working as Designed” may be a good status for a reported bug, but it’s not a very useful status for less-technical things like issues, questions, or action items. For instance, if you create an issue for your team to discuss which payment provider the team should choose, “Working as Designed” simply isn’t a relevant option. Worse, it’s downright awkward. Add in a few more statuses, and things start to fall apart.

If those other data types don’t fit, you’ll likely end up spreading data out into dedicated apps for tracking that information. Once that information lives elsewhere, complexity goes up dramatically. You’ll need to search two locations. You’ll have to constantly go back and forth to link items. Some things will be filed in the wrong application and have to be moved manually. It can get ugly. Having a single location for bugs, issues, questions, tasks, etc. can be surprisingly powerful. Nobody has to ask “where you should I report this?” or “where did we put that?” Everyone just knows.

No Configuration

We’re huge proponents of minimal configuration. There are only a handful of settings in Sifter, and we already feel like that’s too many. The goal of an issue tracker is to get work done, not fiddle with knobs and overclassify everything. In theory, the extra control sounds great, but in reality, it quickly becomes more of a burden than a benefit. Designing a set of statuses isn’t easy. Every team and every situation is different. Spending time, and possibly even meetings, designing a workflow can be a huge distraction.

On the surface, it may seem like simply choosing some words and putting them in order, but outside of the software, workflow is much messier. Issues move forwards and backwards and side-to-side. They get passed around and discussed.

Complimentary Features Fill the Gap

In Sifter’s case, categories, milestones, and projects can be used to handle different statuses. For instance, internally, we keep a separate “Ideas” project that’s kind of like a dumping ground for conversations. Some of those issues eventually get promoted to the “Sifter” project where we’re actually writing code. You can think of that project as a backlog or icebox, but it’s really much more than that. It’s a safe zone for ideas to percolate while we’re staying focused on our active projects.

Similarly, milestones can help determine whether something is under active development. If it’s not in a milestone, it’s not happening. Whether you want to consider that “On Hold” or “Pending” is totally up to you. Another common request is for an “In Progress” status, but to us, that just sounds like more work to explicitly maintain a piece of information that should be implied based on assignment. If an issue is assigned to someone, it’s safe to assume that it’s effectively in progress. If it’s not assigned, or not in a milestone, then it’s not underway.

The Paradox of Choice

On the surface, it seems like additional statuses can only be more helpful. Unfortunately, more statuses also means more complexity which means more choices. It introduces room for statuses to overlap which leads to confusion about which status is the correct status for an issue.

The Sifter interface is explicitly designed around the simplicity of three statuses. Once you add more statuses, that means more colors, more space, and more complexity, which makes it more difficult to create a simple and easily comprehensible interface. The result is added confusion and complexity that gets in the way of getting work done.

An issue is always either going to be open, resolved, or closed. Always. Knowing those three states you can quickly and easily classify where the issue should be. Once you add more layers, it requires a little more thought to classify an issue. Having to stop and think about such things slows things down. The quicker everyone can move issues along the better it is for everyone.

Resolutions are Outcomes, not States

It introduces a temptation to use statuses to classify facets of issues that are best captured elsewhere. For instance, a lot of people want to capture resolutions as a status when resolutions are generally best captured in the description. Things like “working as designed”, “won’t fix”, “user error” all describe outcomes, not states.

On top of that, short resolutions are generally bad resolutions. “Working as designed” doesn’t explain anything or answer any questions. Resolutions should be detailed and helpful. They’ll either help the person reading it now, or they’ll be useful as an in-depth explanation for you two or three months down the line when you need the historical information. Resolutions work best when they’re written. “Closed” isn’t a a resolution. It’s a state that issues arrive at due to a resolution.

Easy Enough for Non-technical Folks

Understanding the different level of statuses becomes even more difficult to the outsider who submitted an issue and wants to track its progress. Open, resolved, and closed, leaves only a little room for interpretation, but when you add more steps, that process becomes a bit cloudy.

As a result, people who aren’t deeply versed in a custom workflow can easily mis-classify things. Or worse, they can be put off from the system simply because the process feels overwhelming. When people aren’t submitting issues, there’s not amount of custom statuses in the world that will help the workflow. Sifter was built to get issues reported and moved through quickly, but that breaks down in a complex system.

Summary

While three statuses won’t work for everyone, we feel that simplicity brings some great advantages. As with any decision, there are tradeoffs, but the majority of the time, for the majority of teams, the benefits of simplicity will far outweigh the potential perceived benefits of additional statuses.

Tracking more than Bugs

2012-08-08T11:45:40-05:00

We refer to Sifter as a “bug and issue tracker”, but really, it’s just a fancy team todo list. You can track bugs, but that’s only the tip of the iceberg. Internally, we dump almost everything into Sifter, and it just might be useful to see how flexible Sifter can be. It’s no accident that Sifter has fixed statuses or that we chose the words “resolved” and “closed” to represent that later stages of an issue.

It’s also the driving reason that Sifter issues have a very simple set of attributes so that they apply equally well to all types. For instance, fields like URL, User Agent, and Operating System aren’t very useful for non-technical matters. Moreover, a URL field only makes sense for web-based software. Of course, just because there aren’t dedicated fields for these pieces of data doesn’t mean that they can’t be added to the description. With full-text search, it’s still just as easy to find them later.

Bugs

Let’s get the obvious one on the table. Sifter is great for tracking bugs. We track all of our own bugs in Sifter. (It isn’t perfect for sharing code just yet, but it’s on the radar.) With bugs, “resolved” represents the state where the bug has been fixed, and “closed” represents the state where the fix has been reviewed, ideally by the original opener, and approved. With some smaller bugs, we might skip retesting and just close them out right away. However, with larger, more complex bugs, we’ll mark them as “resolved” and make sure to review them in staging before we officially close them out.

Issues, Questions, and Discussions

Not everything is a bug that can be solved by changing code. Some things are more generic problems that need to be addressed or just high-level questions that need an answer. For instance, things like pricing or philosophical approaches to problems need a forum to discuss and work towards a discrete outcome. For example, if we’re evaluating a couple of vendors or considering switching vendors, we’ll create an issue in Sifter to handle the discussion. Then, regardless of the outcome, we have a discussion thread to circle back to later.

Ideas

Internally, we even have a dedicated project for ideas. You might think of it as an icebox or a backlog, but it’s really much more than that. We discuss everything from new features to marketing to areas that may need refactoring. Not everything that’s an idea necessarily graduates to be implemented, but it provides a great location to throw out suggestions and see what folks think. (As a side note, this is a little more difficult with larger teams because we don’t make it easy to notify multiple people, but we’re working on that and should be adding @mentions in the very near future.)

New Features & Enhancements

We keep all of the ideas for new features and enhancements in our “Ideas” project. We’ll track discussions and log interesting customer suggestions as well. Even when we decide not to build a feature, we still have a record of our decision and how we arrived at the decision. That way, 6 months down the road when we start to second guess ourselves, we have a clear explanation of our thought process. This gives us the context to either say “well things have changed since then” or “oh yeah, there’s now way we’re going to do this”.

Tasks, Todos, and Action items

Some things aren’t even problems or inherently negative. They’re just tasks that need to be done by a specific person. Anytime something might take a couple of days, we toss it in Sifter. There might not be any discussion about it at all, but it provides a place for updates on every ongoing task. For instance, if an SSL certificate is going to expire in a few months, we’ll create an issue that someone needs to address it. Then, that issue can be updated with status information as the process moves along. If anyone is ever curious about where it’s at, they can hop into the issue and see without having to interrupt the person responsible.

Summary

Sifter doesn’t differentiate any of these types of data internally, because it’s generally not relevant. It’s a distinction without a difference. At the highest level, it’s all simply bits of data that need a formal resolution. It’s work that needs to be done regardless of what “type” it is.

In general, there’s three key points when tracking this stuff. First, it’s about getting data into Sifter, which means making sure that all of the different types of data fit comfortably. If the data isn’t in the app, then nothing else matters.

Second, it’s about enabling a workflow towards a discrete outcome. Issues need to be closed. There’s no “on hold” or “pending” statuses because that’s a way to avoid taking action one way or the other. Our philosophy is that either it’s going to get done and closed as completed or it needs to be closed out right away if it’s not going to happen.

Finally, it’s about a central and searchable place to record decisions. We’ve all had those moments where we look back and wonder “why did we decide not to do this?” or “what was our final decision on that problem?”. Verbal agreements are worthless. With an issue tracker, everything is documented, and it’s always accessible. If your bugs were in one system and your issues or questions or enhancements in another system, that’s not easy to figure out.

Balancing Simplicity with Obscurity

2012-08-06T10:13:58-05:00

By not setting up any categories or milestones by default, many potential customers assumed that Sifter simply didn’t offer them. Not everyone immediately digs into the settings to look for features. We needed a way to let people know that Sifter did offer categories and milestones, but if a team didn’t want to use them, it didn’t make sense to leave a permanent reminder taking up real estate.

So, instead of displaying a constant reminder in place of the category and milestone drop downs, we now only display the suggestion to create categories or milestones during the first 24 hours of a project’s life. That way, the reminder is there when it’s useful, but then it’s out of the way once the project is underway.

Figure 1 The new issue form now displays a sort of “Did you know?” bit about creating categories and milestones in projects. However, it only displays it for the first 24 hours of a new project so as not to get in the way going forward.

Figure 2 The new issue form now displays a sort of “Did you know?” bit about creating categories and milestones in projects. However, it only displays it for the first 24 hours of a new project so as not to get in the way going forward.

We always want to avoid weighing down the interface with clutter, but sometimes that leads to the unfortunate consequence of obscuring or hiding features. We’re always walking a fine line, but this was just one of those situations where a nice simple solution seemed to do the trick and work out great for all parties.