You can add a wildcard entry to the robots.txt like:

Disallow: /addreview*


Google and other big players will honor the wildcards, but as this is a more recent addition to the robots.txt specification, there are probably still crawlers that ignore it.

This will also only work if the URLs you want to disallow have a common element that is not found in URLs you want crawled.

Robots.txt disallowing URLs
I'm working with a fairly complex web application. It's split up into the following layers:


Presentation - HTML
Service layer - A REST and SOAP API communicating with the business layer
Business layer - Contains the business logic.
Data access - Provides access to the storage (SQL etc)


The business layers contains classes encapsulating specific areas, such as customer registration, user management and more. The problem we are seeing is that the business layer is starting to get a bit messy. We have a single class handling customer management but as this area of the application grows more and more complex, the class grows and grows and become messy.

For example, we may have the following classes


class CustomerManager


void CreateCustomer(...)
void DeleteCustomer(...)

class UserManager


void CreateUser(..)
void DeleteUser(...) 
void ActivateUser(...)
void InactivateUser(...)
void ResetPassword(...) 



Creating a customer involves creating users as well. So CustomerManager calls misc methods in the UserManager class. As the application has evolved, creating a new customer means roughly 10 different things needs to be done except for registring the customer in the database, such as informing sales, audit logging, configuring default user accounts, creating a default configuration for the customer, notifying end-users of their auto-generated passwords and more. So CustomerManager.CreateCustomer grows to ~100 lines of fairly hairy code.

I'm trying to think of a good way to handle this but am assuming that there's some common good way to do this which I'm simply not aware of.

I've considered creating "Task"/"Command" classes implementing small sub-processes and then let the CustomerCreation.CreateCustomer simply execute a set of tasks. I would have more classes but they would each do less things.

I've also considering implementing some kind of global application-level event/plug-in systems where CustomerManager.CreateCustomer just creates the customer in the database and then publishes an event that the customer is created. Plug-ins/something can then subscribe to these events and do stuff such as informing sales and logging the fact. Using this method, I wouldn't have to actually update CustomerManager.CreateCustomer when I want to do more stuff which is something which feels attractive to me.

What obvious design pattern am I missing?
