• Hadoop and Map Reduce - Part 3

    The chances are, the most common software you will hear about while you are navigating the Big Data World, is Hadoop.
    Hadoop by itself, is very important and widely used computational platform, and adding to that, it is the base of other software, technologies and applications that are built on top of it.
    Lots, and lots of important technologies, and software in different domains and business are built on Hadoop.
    Hadoop is complex and its computational framework MapReduce is complex, but the good news is there is a good chance that you can use all of the technologies that are built on Hadoop without the need to learn or deal with Hadoop at all.
    In spite of that, still understanding the basic concepts of Hadoop will give you a boost in the Big Data World, becuase you will encounter its concepts in other technologies and software.



    read more ...
    Sending emails with attachments or embedded image using Amazon's AWS SES API & .NET.

    To send an email with AWS services, there are two options, either connecting directly to the AWS SMTP server with SMTP protocols, but then you have to open SMTP ports (587 or 25 usually). Otherwise you can use the AWS API which are bunch of REST API services that you can call them over HTTPS.
    AWS SES API has two options to send emails as described in the documentation:

    1. Send simple formatted email:
      which will make your life easy, but you are limited to send only text messages.
    2. Send raw email in order to send an embedded image or more complicated content.

    In order to send raw email with embedded images, you can just find some code on stackoverflow, but I will try in this post to describe what is going on, and how raw email works, so you can understand the process in details.



    read more ...
    Streaming real data with AWS - Part 2.

    Technology of big data is booming and moving fast, and one of the major sector of big data booming is the requirements to process and analyze real time data that is helping lots of industries using real time data, from IoT, to the analyzing social networks.
    But real time data has its challenges, which the industry answered with many different solutions.
    In this post we are going to cover the technical solutions in AWS for streaming real time data.



    read more ...
    Big Data at your fingertips using AWS services - Introduction.

    Big Data Analytics used to be an activity that require powerful computers and very high specialized skills. But with the cloud computing, you can have huge computing power installed, and configured in few button clicks.
    AWS as an example provide a wide veriety of different kind of data analysis services, each one will do a specific job. We can use these services as building blocks to build any application that deal with big data analysis.
    AWS services shine on big data, and they can handle Peta-byte size databases, with high performance.
    In this post and following posts, I am going to describe these services, and how to use them. At the end, I am going to show how to use different kind of services to build a monitoring and diagnostic tool for an IIS web site application.



    read more ...