Showing posts with label Development. Show all posts
Showing posts with label Development. Show all posts

Sunday, March 02, 2014

ETL plugins for Data Vault

I recently finished a project whose goal was to provide an easy way to load data into a Database modelled using the Data Vault approaches.    At the time, I had chosen an open source ETL tools coming called PDI from Pentaho's suites.

Unfortunately, due to some issues there was a shift of strategy and this tool would no longer be needed.  In the sake of sharing code, I decided to upload this work to Git-Hub, so anyone willing to use/share/improve could do it freely.  Please note, that the code works as specified however a lot more realistic performance test and improvement is expected.  I know for a fact that some level of caching is necessary for the plugin to work in a realistic data warehousing environment.  Anyone feeling up to it, feel free to fork it!

Update note:
The whole project called "DV-loaders" with code source is now moved to github ( https://github.com/mart2010/pdi-datavault-plugin ).


The rest of this post gives more detail on the tools.


DV-loaders provide custom Transformation steps for loading Data Vault compliant objects: Hub, Link and Satellite. These are developed using plugin extension capability offered by Pentaho Data Integration tool (PDI, aka Kettle).

With these custom Steps, you can:
  1. Load quickly and easily HubLink and Satellite
  2. Define as many Steps as needed inside the same Transformation
  3. Load mandatory attributes compliant with logical Data Vault rules
  4. Load non-mandatory fields as-is (ex. batch-Id for auditing)
  5. Adjust buffer size to fine-tune performance
As an example, consider this simple DV data model:
Then one can load it with this simple Transformation:


INTRODUCTION


Data Vault is a methodology suited for building an integration data layer of the Enterprise Data Warehouse that relies on a simple and repetitive pattern-based approach.
ETL tool gives you access to any data input format and any backend output out of the box, without re-inventing the wheel.
PDI is an open source ETL tool that can be extended by custom plugin
Data Vault + ETL + PDI = DV-Loader plugin
DV-Loader plugin makes it even easier and more performant to load any Data Vault model from data stream processed by PDI.

Details


Features

The PDI DV-Loader plugin offers :
  1. Simplicity
    • easy configuration setting using a consistent ETL design-model
    • same process usable for full AND incremental loading
    • Load Hub and Link "expose" technical key (PK) to downstream steps (usable for dependent Satellites)
    • ETL data flow structure closely follows the DV data model
  2. Robustness
    • fully re-entrant and re-startable ETL process
    • support duplicated record in input stream
    • support unsorted satellite historical records
    • support "a posteriori" satellite historical catch-up
  3. Perfomance
    • leverage JDBC batch processing
    • query lookup done not on individual key but rather using a set of keys to minimize the number of sluggish DB round-trip
    • batch and key lookup size is defined by the parameter Buffer size
  4. Compliancy
    • load mandatory DV fields using DV logic:
      • technical keys
      • business keys
      • satellite fields
      • temporal fields setting satellite record validity
      • audit mandatory fields
    • load other none-mandatory field as-is (pass-through attribute like batch-Id for meta-audit)

Assumptions

A very small number of assumptions is assumed:
  • Hub and Link must have primary key defined as unique sequence integer (support any DB data type used for storing integer)
  • Temporal validity of the Satellite rows are determined through a single temporal field (Date type) available in the incoming record stream (entrant hop)
  • Business keys must all be Not Nullable

Rules

The ETL flow design must respect these logical rules:
  • Hub can be loaded once its business key(s) are available in input record stream
  • Link can be loaded once all referred Hub primary keys are available in input record stream
  • Satellite can be loaded once referred hub (or Link) primary keys are available in input record stream (normally appended upstream by Load-Hub step)


USER GUIDE

ETL data flow

Data model is loaded by attaching Load Steps through hops :
  1. Load Hub
    • Connect Step through a hop containing all business/natural key(s) in input stream
    • Step will look-up business/natural key(s) and append the associated tech-key (Hub's PK)
      • When found: append the tech-key returned by DB
      • When not found: generate new key (using sequence method defined) and append it
    • Step will "expose" the tech-key in the output stream as <Table-Name.Techkey-Name>
  2. Load Link
    • Connect Step through hop containing all tech keys of the relationship
    • Step will look-up Hub tech keys and append the associated Link tech-key (its PK)
      • When found: append the tech-key returned by DB
      • When not found: generate new key (using sequence method defined) and append it
    • Step will "expose" the tech-key in the output stream as <Table-Name.Techkey-Name>
  3. Load Satellite
    • Connect Step through a hop containing the Hub or Link's PK tech-key, the attributes and the "From-Date" temporal attribute controlling satellite record's lifecycle
    • Step will load new satellite record based on different use-case:
      • Temporal Satellite with Idempotent=true: sat record is loaded unless there is an identical consecutive records (default)
      • Temporal Satellite with Idempotent=false: sat record is loaded unless irrespective of consecutive records
      • Static Satellite (no "From-Date" temporal attribute defined): sat record is loaded unless one already exist for the Hub

Setting common to all Step

General Properties:
PropertyDefinition
Step nameName of the Step must be unique within the Transformation
ConnectionDatabase connection to use
Hub/Link/Sat tableTarget table name to load
Buffer sizeThis determines the number of input rows processed at once. Too large value may generate Query lookup or batch insert causing JDBC driver error (typical values are of range 500 or more)
Audit-related Properties:
PropertyDefinition
Sys-creation Date/TimeThe column holding the Timestamp indicating when record was loaded into DB (leave empty when not used
Load Record SourceColumn holding the Audit record source information (leave empty if not used)
Record source valueValue to store in the column "Load Record source" (may be set by a variable substitution)

Setting specific to Step "Load Hub"

PropertyDefinition
Attribute MappingMapping between input stream field and database column
Attribute of type Business/Natural Keyfield(s) corresponding to the business key(s)
Other typepass-through field simply loaded as-is (useful for attribute like batch-id, etc.
PK Sequence-Id settingDefine the technical PK column and which sequence generation method to use

Setting specific to Step "Load Link"

PropertyDefinition
Attribute MappingMapping between input stream field and database column
Attribute of type "Relationship Key"field corresponding to the Relationship keys (i.e. each Hub's Primary key)
Other typepass-through field simply loaded as-is (useful for attribute like batch-id, etc.)
PK Sequence-Id settingDefine the technical PK column of the Link and which sequence generation method to use

Setting specific to Step "Load Sat"

PropertyDefinition
Idempotent transformationIdempotent ignores records having identical state at two consecutive time (all attributes are equal). Data Vault standard is Idempotent, but you may have different requirements
Attribute MappingMapping between input stream field and database column
Attribute of type "Foreign-Key to Hub"field representing the Hub's PK (typically provided by an upstream "Load Hub" step)
Attribute of type "From-Date Temporal"field representing the temporal attribute controlling the timeline of each Satellite record. Using an input field instead of a fixed adds more flexibility: if is appropriate, then we simply append it upstream, but other attribute could also be used (ex. when using file input, when using table input, etc..). You use the temporal attribute most adapted for your use-case
Attribute of type Normalfields recorded inside the Satellite. All these control sat record timeline (or lifecycle). Pass-through fields may only be added if their changing values do not impact Satellite lifecycle with regard to the Hub record (the ETL batch-id attribute is a valid example of this)


INSTALLATION

Pre-requisite

 PDI version 5.x

Download and Install

* Download latest archive/package
* Unzip it inside folder: ${PDI_HOME}/plugins/steps

Check installation

* new folder 'DV-loader' should now exist in: ${PDI_HOME}/plugins/steps
* Re-start PDI Spoon UI (${PDI_HOME}/spoon.sh or ${PDI_HOME}/spoon.bat)
* Create a PDI Transformation and add DV-loader steps found under category Experimental:
  • Load Hub
  • Load Link
  • Load Sat

Martin

Sunday, April 27, 2008

Decorating RPC server call

In RPC service layer, I often need special processing to occur before/after some of the call methods. For example, I may want to gather various access statistics related to some of the methods, compress the response when it exceeds some maximum threshold, or yet simply providing a cache response to speed things up!

So in essence, I need to intercept specific RPC calls to provide extra service to these calls and optionally even taking responsibility of handling the call without reaching the real service layer (e.g. as in the case of caching). The logical answer to this need would be to use a AOP solution (Aspect-Oriented Programming), however my service layer are already intercepted by some aspects to manage orthogonal concern such as transactional processing and security handling. And defining additional aspects around my service methods seem counter-intuitive especially for aspects that are relatively sparse (i.e. only apply to a few methods/service).

My solution to this problem is simply leveraging the Decorator design pattern. Conceptually, this pattern provides a way to attach additional responsibilities to object dynamically, i.e. here I could add auditing ability or caching ability to the service object. The next UML figure presents the class diagram involved in my solution. It is a slightly modified version of the classic pattern because both my concrete service object (to be wrapped by decorator) and my decorators inherit the same super class. This is more a convenience to centralize the various steps originating from GWT RPC handling mechanism with the template method pattern. The drawback is that all decorators have an annoying dependence on the GWT library. This is fine for my scenario, but I may have to refactor this in order to improving testability.








I’m providing the skeleton source code for the main classes involved, feel free to adapt them to your needs.
(no longer has access to my uploaded file, so I just removed these for now)


Martin

Friday, November 23, 2007

Web development

Up until my current project which uses GWT to develop a rich web client, I always followed the best practices in terms of Web development. I used to develop thin and dumb client view (e.g. jsp) which were fed and render based on data model objects and eventually sent back to the web client.

All web requests are taken care by some dedicated controller (or action in Struts parlance) that are responsible in analyzing the user request, delegating the request to the service layer (the core of the application where business rules usually sits), obtaining the return domain object from the service and preparing the response by selecting the proper view to be rendered and sent to client. Ok, that is a very brief and incomplete description of the MVC pattern adapted to the Web server and HTTP protocol. Although this design pattern may be overkill for smaller applications, the advantages (e.g. makes web applications that are easier to maintain and are logicially divided into layers that are inherently decoupled and whose focus are on very different software concern) usually outweight the additional learning curve and relative complexity.

A great deal of coding must be done on the controller aspect of the MVC. And a lot of this includes details originating from handling the request/response of the stateless HTTP protocol...which has very little to do with the main application business logic. If I'd have the choice, I'd rather spend this time developing and improving the business objects domain, the related business rules logic and the database model, i.e. my preferred subjects and expertise!


With the advent of AJAX Web 2.0 fat client, a great deal of the weight that used to sit on the web server shoulder implementing this MVC pattern has now moved toward the client sitting on the other side of the communication channel. Web clients now make asynchronous RPC (remote procedure call) to the server without having to deal with the of HTTP concern details (for example the RPC mechanism of the GWT API isolates client code these). They can also now handle their own session state by keeping and handling the data model locally.

The server can focus less on the "C" of the MVC, and more on the service core layer designed to enforce and apply all business rules logic. The client code can choose to implement the traditional MVC pattern, when its complexity justifies it.

In my current project, all web controller codes disappeared and my web application does not move anymore from one page to another (following subsequent user request), but rather change its unique page appearance (by replacing/changing section of page through DOM manipulation) followed by user action and RPC response from the server.

My next post will present the way I've implemented my RPC client code to talk directly to my service layer using the Spring Framework. Now, I need to get back to work!

Martin

Sunday, October 14, 2007

my Web2.0 experience with GWT

My last month as a freelancer allowed me to have made quite important progress on my current project. I find it quite motivating since I'm the main analyst/designer/developer/tester and even business analyst (actually, I'm all alone on that one, so no one to blame or complain about;-). Technology wise, it's a perfect situation since I can freely chose all components based on design merit and no decision is based on political constraint of some sort.

Currently the technologies involved are:

On the server:

- Spring framework, iBatis, ehcache, Quartz, Velocity, and eventually the servlet container (not yet decided.. but the development is obviously done using Tomcat).

On the client:

- GWT and a few velocity pages.


The novelty for me is the use of GWT (Google Web Toolkit). I've already mentioned my frustration in developing client side code especially using http-based web client. The time spent on the page design messing around with jsp tag, html and css styling seems to me as a big wasted time and effort. That's why in my professional life I'm usually involved muck closer to the back end where my expertise is more profitable for the client.

Now with the advent of GWT, client side component is no longer restricted to thin and dumb page presenting the view/model already prepared on the server. A lot can happen on the client with the capability to hold on great deal of state information and business logic. And this magic is done without adopting Javascript and sacrificing the strongly-typed property of Java.... the toolkit does the Java-->Javascript conversion for you!!!

So far I'm quite pleased with the toolkit as it practically eliminate the need to write any html markup... however all styling still depends on css. I'm probably half-way there for the client part and only have a single html page with just a few lines of markup html code.

The server integration is done through RPC call (done asynchronously à la AJAX style) with servlet on the server end. A great deal of design effort is done by keeping the server scalability as a top priority. Because the application will target the web public at large, most design decisions are affected by this, e.g.

  • Keeping minimal state on the server. As a matter of fact, with the help of GWT most state is kept locally while the server only keeping a session token (cache in memory) to identify valid client connection. This will avoid issues involved when relying on HttpSession object (to hold client state information within servlet container) such as server affinity and more importantly larger memory footprint per connecting session.



  • Data is cached both on the client side and on the server side. Because some data is not mission critical, the client keeps some data content (limited size) and the server keeps all data content in cache eliminating the need to hitting the database back-end during most client request.



  • The cache server data content is refreshed at fixed interval of time which is configurable depending on the data volume and the database query extraction time



  • The cache server data content is kept in a de-serialized GWT form, avoiding the need to serialize the payload of server response at each client request.


I'm confident to be able to put a beta version of the application before the end of year... stay tuned!



Martin

Friday, September 29, 2006

I hate UI-type development

Why do I hate having to deal with UI-type development... I don't know but I have some hints:

I simply suck at it!

Although I enjoy using intuitive UI and appreciate the design value of it, I neither have the patience nor the talent to do it! In my view so much time and effort spent simply in designing a nice HTML/JSP page or rich client equivalent (with SWT for ex.) is too frustrating for the end result. I sometimes have to do it when delivering an end-to-end product for clients, and typically most of my time will be wasted on these UI stuff! I guess I could outsource all these, actually I even tried it once... but finding a good designer willing to develop inside JSP page is another challenge on its own!!!

When I first did some RCP stuff in Eclipse, I appreciated all the advanced design patterns available in library such as JFace but I soon got bored and tired again in dealing with all these widgets details consideration, I'm hopeless.

I guess I'll stick to creating domain business layer, service business layer, data access layer, and other more non-visual feature!

Martin

Tuesday, September 12, 2006

Unit Testing but...

I really appreciate developing using the unit testing approach and as such I always have a JUnit library somewhere in my classpath while building an application. This really brings up my confidence into my code and allow me to refactor at ease without worry about breaking all existing functionality.
However, there are a few stricter recommendations commonly found among the unit tester fanatic or the extreme programmer advocate that I find, to say the least, debatable:


  1. your unit test should focus on a single method of a single class, otherwise it is not really unit test.
  2. always build your unit test first and then your application class afterward.

Point 1 actually emphasizes the term unit, and violating this makes your unit test more like integrated test which I agree. But in my view these tend to be more meaningful and practical.

First of all, methods should be small, precise, have a single clear responsibility and have descriptive name that conveys their purposes. As such I tend to agree with recommendation that limit the size of a single method (R. Johnson gave as a ballpark figure between 30-40 lines of code including all comments, while J. Kerievsky goes as far as recommending ten lines of code or fewer, with the majority of them using one to five lines of code). Keeping methods small and giving them intuitive name produce much easier and self-documented code: I like this idea since it helps reduce the need for documenting your code!

This is why I feel that the principle 1. above is opposed to the "writing short method" approach, since small method do not contain enough complex logic that requires a dedicated unit test on its own.


A junit class that test and validate the effect of each and every single method produces on the state of current object or some other dependents (through Mock-up objects) is often straightforward and thus overkilled! Also, a large number of method may not deserve a full dedicated test on them, since not only their logic is simple but also the impact on state is minimal.

That's why I twist my unit test a bit to make them more integrated test, i.e. test only important methods in the class in relation with their impact on itself and on its dependencies (external library, other piece of my code..). Ok, this is not always possible especially when the dependency library is costly and resource intensive component (then I'll use Mock-up for such case), but in very frequent usage, this allows me to validate and better understand the external library during my test as well as testing my code against its dependency. I find myslef even doing such integrated test with code at the service layer level (above the DAO layer) and validating its effect at the database tier. Using a small memory-based database engine such as HSQLDB helps negating the perfomance penalty of doing this.

As for the point 2, I usually adopt more of a concurrent approach, i.e. draft the application class and once it stabilizes create the test class and making it evolve simultaneously. The first few version of my class/interface are a bit too dynamic and sketchy to really have an accompanying test class. So to limit the need to duplicate my changes in both, I'd rather wait till I'm more comfortable with the class/ interface and then proceed with writing test case.
The only advantage I see in creating the test case first, is when I really don't know how my object's going to be used in the client code. However, in that case, I'd rather use a pencil and sketch some use case scenario beforehand...

Martin

Thursday, August 03, 2006

Java and Oracle

Oracle has been committed since Oracle8i in integrating Java within its database/application architecture. Being confronted to the development of a particular application highly tied to Oracle, I'm taking this opportunity to review the current state of affair as of Oracle 10g. Here's what I found:

Originally the strategy was to follow a database-centric strategy where merely all software layer would be offered and hosted directly inside the database engine. This controversy strategy (to say the least) has since been reversed from 9i and 10g where some J2EE technologies already integrated inside the database (e.g. EJB container, JSP and servlet) have been desupported.

The focus is now on providing a complete Application Server suite (J2EE compliant) outside the database offering a vast number of services and support, pretty much like IBM WebSphere, BEA WebLogic or JBoss Application Server.

However, this strategy leads to the development (from beginning of Oracle 8i) of a fully functional and compatible Java Virtual Machine inside the database: OracleJVM.

Each of these two components are commented next.


1- OracleJVM

As of 10g release the OracleJVM offers these characteristics:

  • support the J2SE 1.4.2 as specified by Sun Microsystems
  • supports only the headless mode of the Java AWT (i.e. no GUI will be materializable on the server or remotely)
  • java classes (bytecode), resources files and java source code (optional), all reside at the database and stored at the schema level (knows as the Java schema object)
  • each session (user connecting to the database and calling Java code) will see its own private JVM (although for performance reason the implementation does share some part of Java library between session)
  • core Java class libraries are run natively through the use of Ahead-of-time compilation to platform-specific C code before runtime
  • core Java libraries are stored and loaded within the PUBLIC schema and thus available to all other schema
  • application specific Java classes are stored and loaded within the user schema (the owner)
  • besides writing the Java class, compiling it and running it, OracleJVM requires two extra steps in its development/deployment cycle: 1- class needs to be loaded into the database (done through a utility called loadjava, 2- class needs to be published when callable from SQL or PL/SQL (done by creating and compiling a call specification or a.k.a. PL/SQL wrapper) to map the Java's method parameter and return type to Oracle SQL type.
  • granting execution rights is also needed when running a Java classes located in other user's schema
  • class loading is done dynamically as in conventional JVM, however it is done into shared memory, so only one-time loading speed hit is encountered among all users code requiring the class
  • instead of a global classpath defined at runtime to resolve and load all application classes, OracleJVM uses a resolver per each class during class installation specifying in which schema their depending classes reside
  • multi-threading is usually achieved using the embedded scalability of the database server, making Java language-threads needless since they won't help improve the concurrency of the application (this helps avoid complex multi-threading issue inside Java code)
  • OracleJVM offers adapted version of JDBC (called server-side internal driver) which is specially tuned to provide fast access to Oracle data from Java stored procedure, as well as a optimized SQLJ server-side translator.

Execution control:

How do we exactly start off a Java application located inside the Oracle database or in other words what is the equivalent entry point of the static main method in a "normal" application launched by a conventional JVM? This process is referred to in Oracle terminology as a Call and can be done by calling any static method within available loaded and published classes. These published classes must then contain a static method entry point, and are qualified as the Java counterpart of a PL/SQL procedure (referred to by the term Java Stored Procedures).

Some possible scenario of a Java called includes:

  1. a SQL client program running a Java stored procedure
  2. a trigger (i.e. event fired off by defined SQL DML statement) running a Java stored procedure
  3. a PL/SQL program calls a Java code

These Java Stored Procedures are callable from PL/SQL code but can also call PL/SQL procedure.

Some thoughts: Even though I've never played with OracleJVM, I'm yet to be convinced about its advantage: stored Java procedures seems a bit like writing Java code with a procedural mindset? It seems that the only advantage is the possibility to write and centralize business rules that are more portable and powerful than PL/SLQ code and that are available to application written to bypass the Application Server tier?

2- Oracle OC4J J2EE Application Server (or a.k.a. OracleAS):

This server referred to as OC4J, now includes an ever growing number of components (Web server, J2EE technology, ORM with TopLink, Portlet, wireless, Business Intelligence, etc). Its J2EE support includes: JSP, servlet, JSF and ADF framework (using event-based model for web http processing), EJB, JNDI, XML support (schemas, namespace, DOM, SAX, XPath...), Web Services (WSDL, UDDI, SOAP).

The type of applications supported by this infrastructure are usually large and complex, i.e.

  • involve multiple application tier: the central AS tier where the business logic is maintained, a web tier (maybe part of the AS tier) interacting with Web clients, a backend database tier where persistent data is preciously stored, client tier from fat to thin.
  • involve multiple user with different role and rights accessing concurrently common data
  • involve different remote user sites (i.e. implies Web access) and heterogeneous environment
  • involve sophisticated business rule;
  • involve interaction with other EIS enterprise information system through the J2EE connector Architecture (ERP such as SAP, legacy information system)
  • involve web services support

Of course not all application will need all these, but to pull its weight and leverage this considerable software infrastructure weight, the application specification should meet a fair level of complexity before committing to this framework. This technology overweight is probably responsible of the creation of lighter and simpler initiative coming from opens source community (lightweight Framework only requiring a web jsp/servlet container, such as the one I described here).

Martin

Monday, May 29, 2006

J2EE development

Before starting to do any J2EE Web development I did my own research on tools and libraries that would best meet my web transaction-based application requirement (e.g. things like flexibiltiy, simplicity, availability, cost, adoption..). I finally decided to go with the Spring Framework for all integration code and parameterization settings, Hibernate on the data tier to handle the ORM aspect, and Struts for the Web tier. I discovered since then that these exact set of tools are promoted by Source Labs (http://www.sourcelabs.com/?page=software&sub=sash) as the SASH stack. Although I appreciated developing using these libraries, I enjoyed even more the best practices that these frameworks encourage through the adoption of sound principle: loose coupling between component, seperation of concerns, design pattern uage like MVC or dependency inversion, etc.


You have a feeling when you're building applications along these principles that it is well architect and clean, however you enjoy it even more when 5-6 months later the client calls you to update its requirements!

Without going into details, the web application developped with these frameworks usually follows architecture along these lines:
  1. A seperate and "dumb" View layer (JSP page);
  2. A seperate Control layer (action and action setting files in Struts)
  3. A separate Model/Business layer (business layer is using simple POJOs following java bean rules which allow dependency injection with Spring)
  4. A seperate Data layer (thourgh DAO and Hibernate ORM)
  5. An integration and configuration to glue all layers through Spring bean application context file
  6. And finally a simple servlet/JSP container server ( e.g. Tomcat) to service and host the application deployment.

As a good advocate of open source, I put my principle into practice by putting such an application available to anyone intrested, just contact me by email and I'll send you a copy of the project.


Martin

Friday, December 16, 2005

Design vs Coding, and DDD

I have the chance (some would probably not call it that way) to deal with architecture/design stuff in my work, in addition to doing real coding and hands-on development work during my consulting activities.

For me, it's sort of an experimental way aiming to find which of one of these two would fulfill my day the most. After doing the two for the past 2 years: I came up to the conclusion: the mix of both is what fits me since coding allow to stay more in touch with the technical aspect and the technology while the design and architecture helps me keep some perspective as to which technology can make some difference and which adds up to ... well mostly hype!

An interesting approach that I find in OO-type of development that actually stresses the importance of having both competence in order to build complex system having all usual quality indicators. This approach originated from Eric Evans and is now well accepted and referred to as Domain-Driven Design (or DDD). I will not describe it here, please refer to his book.

However let me just comment of one aspect that I find particularly useful during OO design of a typical business application. It really helps in producing well organized, well designed, well documented and logical Domain Model construction. Very briefly the way I see this particular concept, is similar to a form of taxonomy that helps you organize and classified each object that you design in your Domain Model based on their roles:



  • Entity Objects are those with a very precise identity and transactional lifecycle, and for which we usually worry when multiple instances of the same identity could be found at runtime. They would typically be mapped to a unique row in the persistence database.



  • Value Objects are those without any identity where its field attribute are only compared by value (i.e. two instances with same value are considered equal).



  • Factory Objects are those concerned with the creation of object(s)... more often justified when creating group of objects along with their associated graph. This is required when this creation involve enough complexity to warrant encapsulation through factory.



  • Service Objects are those used to handle and implement business service along with associated business rules. These service objects usually operate on multiple collaborators entity objects as well as their repository.

  • Repository Objects are those used to handle entity lookup and persistence through an abstract layer hiding any details and idiosynchracies of the related persistence engine. They are typically used by Service objects, but sometimes by Entity Object, in which case Entity object would have a reference to the Repository passed inside its methods requiring Repository functionality.
I find this taxonomy very insighful and convenient, and this is certainly useful to adopt..still worth it when only used to help standardizing design terminology among developers and architect!


Martin