We have started working with Concept Soft on the development of the management system for video analysis and editing. The experience has exceeded our expectations and we were satisfied with the quality of the job. The team we worked with pleased us with their professionalism, dedication, responsiveness and forward thinking approach.
Our cooperation continues and we are confident that our business relations will strengthen. Concept Soft is the right company to deal with thus we highly recommend them and will make business with them again.
DealFinder application development
Customer: Under NDA
Solution: upon detailed analysis of the requirements, our team has outlined the following technical challenges for the development:
- eBay data processing;
- advanced search capabilities;
- further supplies comparison;
- system high load resistance;
- high productive, easy scalable system, ready for evolution, flexible tuning and self-monitoring.
Among the other prime criteria was the requirement of one second search over the base.
Concept Soft specialists have developed the back-end layer using modularized easy scalable architecture.
Based on client's requirements, we have used NoSQL DBMS Apache Cassandra for operating with large amount of data for the following benefits:
- fast processing,
- easy scalability with redundancy,
- advantages of MapReduce pattern use.
The developed system is powered with Solr search (including geo search) and Hadoop for analysis and data transformation jobs. System job management and scheduling is Quartz based. We have implemented business logic scalability by the advanced use of Terracotta Cluster and its Master-Worker additional module. Cache performance optimization was leveraged by utilization of Terracotta clustered Ehcache. To bond key technologies and business logic Concept Soft engineers have used Spring framework.
Self-monitoring and benchmarking for bottle-neck detection in production environment makes system more predictable and stable. Cluster configuration results in 2 types of nodes easy for on-demand adding. For load balancing and high availability HAProxy was used.
Selecting Amazon EC2 proved to be a cost-efficient decision, as computing capacities are paid based on consumed instance hours; moreover the platform provides possibilities for easy application scaling. In case the use of Amazon is no longer required, the system may be seamlessly ported on customer's physical cluster.
Technologies: Java, Amazon AWS (EC2, EBS, S3), eBay API, Spring (IoC, AOP, RESTful WS, RMI), Apache Cassandra 0.7, Solr, Apache Hadoop, Terracotta 3.5 (including Distributed Ehcache, Quartz 2.0 scheduling), HAProxy, Tomcat 7.0.