Thursday, December 29, 2016

Exception

[19:00:47] DEBUG impl.execchain.MainClientExec  - Connection can be kept alive for 60000 MILLISECONDS
[19:00:47] DEBUG impl.conn.PoolingHttpClientConnectionManager  - Connection [id: 0][route: {s}->https://dynamodb.ap-northeast-1.amazonaws.com:443] can be kept alive for 60.0 seconds
[19:00:47] DEBUG impl.conn.PoolingHttpClientConnectionManager  - Connection released: [id: 0][route: {s}->https://dynamodb.ap-northeast-1.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
Exception in thread "main" java.lang.UnsupportedOperationException: ListIterators are not supported for this list
    at com.amazonaws.services.dynamodbv2.datamodeling.PaginatedList.listIterator(PaginatedList.java:459)
    at java.util.List.sort(List.java:479)
    at java.util.Collections.sort(Collections.java:175)
    at me.otta.backend.repository.datastore.LocationHistoryRepository.getLocationHistoryData(LocationHistoryRepository.java:124)
    at me.otta.backend.main.Application.main(Application.java:185)

DynamoDB indexing

What about indexes? Well in a RDBMS, any field or combo of fields you’ll be querying – you want to index (though this depends on the RDBMS you use, and how exactly you want to index). In Dynamo, there are a number of limitations on “local secondary indexes” (LSI). First and foremost is that you can only have LSIs if you have a Hash and Range primary key. The second limit is that LSIs are single attribute only (DynamoDB calls them “attributes”, RDBMS would call them a column). Third, you can only have 5 LSIs per table. Lastly, you cannot add/modify/remove LSI after you’ve created the table. Since this is NoSQL, all attributes beyond your primary key are flexible – but if you want to use LSI, plan carefully.

Resource Link:

Wrapping my brain around DynamoDB

Friday, December 23, 2016

DynamoDB pros and cons


DynamoDB is awesome, but…




So. A few friends and I got together a while ago to build an app for the newly launched APIs for Mxit. We didn’t plan on making it big, just having some fun. So we also decided to learn new stuff across the board. I’ve worked with Heroku in the past, but never EC2. So we got a completely new stack going: EC2, Node.js and DynamoDB.
The first thing we are running into that isn’t panning out as we had hoped is DynamoDB.
Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability.
You pay for throughput. Amazon does the rest. They replicate the data, managed it, and let it scale automatically. If you need more throughput (reads/writes per second) you simply dial it up. This itself is quite awesome and clever.

However we have run into some problems that are a pain to iron out. Hopefully by telling someone about it, it can be avoided.

There are 4 ways to fetch items from DynamoDB: Get, BatchGet, Query and Scan.
  • For Get and Query, you can only use the hash key (think of primary key in sql dbs) and the range key. 
  • If you want to fetch items based on any other values, you have to use a scan and add your own conditions (here you can now use anything, not just the hash and range key). The problem is: Scan is expensive. It reads the entire table and fetches what is relevant.

Cool. For now, we aren’t worried that much about performance. We are just playing around after all. WRONG. You can’t sort on Scans! FFFuuu.

You can only fetched sorted results based on the range key you specified in the table (ie one column) with a Query call. Not really knowing we had to do that (our own stupidity) means not one of our tables have a range key. You can’t retroactively add one. The only way to assign a new range key is to create a new table, port over all the data. Now wanting to fetch any type of sorted data, we have to put aside a whole afternoon to code in the table migration, etc. Not ideal.

Chris Moyer has a solution to this. Create a new table with new hash and range key referencing the hashes of the parent tables. Now you can ‘sort’ on any column. The problem is, you have to open up new tables, provisioning throughput to them too, costing more money.

We have to do something about it soon, because we have anyway been hitting our throttledrequests (probably due to all the scans).




So, we have to decide now. Are we sticking with dynamodb, or moving to our own mongodb cluster? I think for our needs right now, dynamodb is causing more headaches than not. We haven’t hit mass scale yet where dynamodb’s provisioning model will help us, and we don’t want to be restricted by how dynamodb handles the data.

So if you want to use dynamodb, remember that!

Resource Link: http://simondlr.com/post/26360955465/dynamodb-is-awesome-but

DynamoDB Query Filtering

Query Filtering

DynamoDB’s Query function retrieves items using a primary key or an index key from a Local or Global Secondary Index. Each query can use Boolean comparison operators to control which items will be returned.
With today’s release, we are extending this model with support for query filtering on non-key attributes. You can now include a QueryFilter as part of a call to the Query function. The filter is applied after the key-based retrieval and before the results are returned to you. Filtering in this manner can reduce the amount of data returned to your application while also simplifying and streamlining your code.
The QueryFilter that you pass to the Query API must include one or more conditions. Each condition references an attribute name and includes one or more attribute values, along with a comparison operator. In addition to the usual Boolean comparison operators, you can also use CONTAINS, NOT_CONTAINS, and BEGINS_WITH for string matching, BETWEEN for range checking,  and IN to check for membership in a set.
In addition to the QueryFilter, you can also supply a ConditionalOperator. This logical operator (either AND or OR) is used to connect each of the elements in the QueryFilter.

Resource Link:

https://aws.amazon.com/blogs/aws/improved-queries-and-updates-for-dynamodb/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+AmazonWebServicesBlog+%28Amazon+Web+Services+Blog%29

Read it

Thursday, December 22, 2016

40 Interview Questions You Should Be Prepared to Ask & Answer

When query is better than scan in aws dynamodb?

Last week, Amazon announced the launch of a new product, DynamoDB. Within the same day, Mitch Garnaat quickly released support for DynamoDB in Boto. I quickly worked with Mitch to add on some additional features, and work out some of the more interesting quirks that DynamoDB has, such as the provisioned throughput, and what exactly it means to read and write to the database.

One very interesting and confusing part that I discovered was how Amazon actually measures this provisioned throughput. When creating a table (or at any time in the future), you set up a provisioned amount of "Read" and "Write" units individually. At a minimum, you must have at least 5 Read and 5 Write units partitioned. What isn't as clear, however, is that read and write units are measured in terms of 1KB operations. That is, if you're reading a single value that's 5KB, that counts as 5 Read units (same with Write). If you choose to operate in eventually consistent mode, you're charged for half of a read or write operation, so you can essentially get double your provisioned throughput if you're willing to put up with only eventually consistent operations.

Ok, so read operations are essentially just look-up operations. This is a database after all, so we're probably not just going to be looking at looking up items we know, right?

Wrong.

Amazon does offer a "Scan" operation, but they state that it is very "expensive". This isn't just in terms of speed, but also in terms of partitioned throughput. A scan operation iterates over every item in the table, It then filters out the returned results, based on some very crude filtering options which are not full SQL-like, (nothing close to what SDB or any relational database offers). What's worse, a single Scan operation can operate on up to 1MB of data at a time. Since Scan operates only in eventually consistent mode, that means it will use up to 500 Read units in a single operation (1,000KB items/2 (eventually consistent) = 500). If you have 5 provisioned Read units per second, that means you're going to have to wait 100 seconds (almost 2 minutes) before you can perform another Read operation of any sort again.

So, if you have 1 Million 1KB records in your Table, that's approximately 1,000 Scan operations to perform. Assuming you provisioned 1,000 Read operations per second, that's roughly 17 minutes to iterate through the entire database. Now yes, you could easily increase your read operations to cut that time down significantly, but lets assume that at a minimum it takes at least 10ms for a single scan operation. That still means the fastest you could get through your meager 1 Million records is 10 seconds. Now extend that out to a billion records. Scan just isn't effective.


So what's the alternative? Well there's this other very obscure ability that DynamoDB has, you may set your Primary Key to a Hash and Range key. You always need to provide your Hash Key, but you may also provide the Range Key as either Greater then, Less then, Equal To,  Greater then or equal to, Less then or equal to, Between, or Starts With using the Query operation.

Unlike Scan, Query only operates on matching records, not all records. This means that you only pay for the throughput of the items that match, not for everything scanned.

So how do you effectively use this operation? Simply put, you have to build your own special indexes. This lends itself to the concept of "Ghost Records", which simply point back to the original record, letting you keep a separate index of the original for specific attributes. Lets assume we're dealing with a record representing a Person. This Person may have several things that identify it, but lets use a unique identifier as their Hash key, with no Rage key. Then we'll create several separate Ghost records, in a different table. Lets call this table "PersonIndex".


Now if we want to search for someone by their First Name, we simply issue a query with a Hash Key of property = "First Name", and a range Key of the first name we're looking for, or even "Starts With" to match things like "Sam" to match "Samuel". We can also insert "alias" records, for things like "Dick" to match "Richard". Once we retrieve the Index Record, we can use the "Stories" property to go back and retrieve the Person records.

So now to search for a record it takes us  Read operation to search, and 1 Read operation for each matching record, which is a heck of a lot cheaper then one million! The only negative is that you also have to maintain this secondary table of Indexes. Keeping these indexes up to date is the hardest part of maintaining your own separate indexes. however, if you can do this, you can search and return records within milliseconds instead of seconds, or even minutes.


How are you using or planning to use Amazon DynamoDB?


Resource Link: http://blog.coredumped.org/2012/01/amazon-dynamodb.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+ChrisMoyer+%28Chris+Moyer%29

  1. http://dynamo.jcabi.com/example-query-scan.html

Wednesday, December 21, 2016

Faisal Vai's Blog

Scan Vs Parallel Scan in AWS DynamoDB and Avoid Sudden Bursts of Read Activity

Scan Vs Parallel Scan in AWS DynamoDB and Avoid Sudden Bursts of Read Activity

1. How scan works in AWS DynamoDB?
Ans:
i) Scan operation returns one or more items.
ii) By default, Scan operations proceed sequentially.
iii) By default, Scan uses eventually consistent reads when accessing the data in a table.
iv) If the total number of scanned items exceeds the maximum data set size limit of 1 MB, the scan stops and results are returned to the user as a LastEvaluatedKey value to continue the scan in a subsequent operation.
v) A Scan operation performs eventually consistent reads by default, and it can return up to 1 MB (one page) of data. Therefore, a single Scan request can consume
(1 MB page size / 4 KB item size) / 2 (eventually consistent reads) = 128 read operations.
2. How parallel scan works in AWS DynamoDB?
Ans:
i) For faster performance on a large table or secondary index, applications can request a parallel Scan operation.
ii) You can run multiple worker threads or processes in parallel. Each worker will be able to scan a separate segment of a table concurently with the other workers. DynamoDB’s Scan function now accepts two additional parameters:
  • TotalSegments denotes the number of workers that will access the table concurrently.
  • Segment denotes the segment of table to be accessed by the calling worker.
iii) The two parameters, when used together, limit the scan to a particular block of items in the table. You can also use the existing Limit parameter to control how much data is returned by an individual Scan request.
3. Scan vs Parallel Scan in AWS DyanmoDB?
Ans:
i) A Scan operation can only read one partition at a time. So parallel scan is needed for faster read on multiple partition at a time.
ii) A sequential Scan might not always be able to fully utilize the provisioned read throughput capacity. So parallel scan is needed there.
iii) Parallel Scans, 4x Cheaper Reads.
4. When Parallel Scan will be preferred?
Ans:
A parallel scan can be the right choice if the following conditions are met:
  • The table size is 20 GB or larger.
  • The table's provisioned read throughput is not being fully utilized.
  • Sequential Scan operations are too slow.
5. Is filter expression is applied before scan?
Ans: No, A FilterExpression is applied after the items have already been read; the process of filtering does not consume any additional read capacity units.

1. http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Scan.html
2. http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/QueryAndScan.html#QueryAndScanParallelScan
3. https://aws.amazon.com/blogs/aws/amazon-dynamodb-parallel-scans-and-other-good-news/
4. http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/QueryAndScanGuidelines.html#QueryAndScanGuidelines.BurstsOfActivity
5. http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ScanJavaDocumentAPI.html#DocumentAPIJavaParallelScanExample

Another blog link
  1. https://egkatzioura.wordpress.com/2016/10/03/scan-dynamodb-items-with-dynamodbmapper/

Query and Scan: Find replies to a forum thread posted in the last 15 days using DynamoDBMapper

wire-level logging turned on

Monday, December 19, 2016

Is there a way to collapse all code blocks in Eclipse?

There is a hotkey, mapped by default to Ctrl+Shift+NUM_KEYPAD_DIVIDE.
You can change it to something else via Window -> Preferences, search for "Keys", then for "Collapse All".
To open all code blocks the shortcut is Ctrl+Shift+NUM_KEYPAD_MULTIPLY.
In the Eclipse extension PyDev, close all code blocks is Ctrl + 9
To open all blocks, is Ctrl + 0


Resource Link: http://stackoverflow.com/questions/1726525/is-there-a-way-to-collapse-all-code-blocks-in-eclipse

HTTP Status 405 - Request method 'POST' not supported (Spring MVC)

Check if you are returning a @ResponseBody or a @ResponseStatus
I had a similar problem. My Controller looked like that:
@RequestMapping(value="/user", method = RequestMethod.POST)
public String updateUser(@RequestBody User user){
    return userService.updateUser(user).getId();
}
When calling with a POST request I always got the following error:
HTTP Status 405 - Request method 'POST' not supported
After a while, I figured out that the method was actually called, but because there is no @ResponseBody and no @ResponseStatus Spring MVC raises the error.
To fix this simply add a @ResponseBody
@RequestMapping(value="/user", method = RequestMethod.POST)
public @ResponseBody String updateUser(@RequestBody User user){
    return userService.updateUser(user).getId();
}
or a @ResponseStatus to your method.
@RequestMapping(value="/user", method = RequestMethod.POST)
@ResponseStatus(value=HttpStatus.OK)
public String updateUser(@RequestBody User user){
    return userService.updateUser(user).getId();
}

Resource Link: http://stackoverflow.com/questions/11145884/http-status-405-request-method-post-not-supported-spring-mvc 

Friday, December 2, 2016

Amazon S3 - How To Create User And Grant Permissions To Access And Manage Account Files

AWS Policy Generator

Design Pattern: great tutorial for beginners...

Best Tutorial: Hello World Web Service Example

SOAP: Generating Stub Code for a Java Client

Generating Stub Code for a Java Client

Use the following steps to generate the stub code for a Java web services client application, using the wsimport tool that is included with the Java Development Kit.
  1. Open an xterm or command prompt window and change to the directory that contains the wsimport program, usually JDK_install_dir/bin/.
  2. At the command prompt, type wsimport -s source_dir -d classes_dir URL_TO_WSDL and press Enter.
    For example, to generate stub classes for the SAAS version of the Named Resource Service, you might enter the following command:
    wsimport -s output/source -d output/classes http://dev.pb.com/NamedResourceService/services/NamedResourceService?wsdl
After the command has been executed, the generated source .java files are placed within the directory you specified with the -s option, and the compiled .class files are placed within the directory you specified with the -d option.

AWS various access rule

  1. Bucket Object expires after 24 hours:

IAM Policy Warning

Please make sure that if you use this code you create an appropriate IAM account policy to prevent mis-use. Example, a policy like the following would only allow PUT access to the bucket for a specific IAM user. You could also set the bucket objects to automatically expire after 24 hours which would prevent people flooding your account.
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt126637111000",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject"
      ],
      "Resource": [
        "arn:aws:s3:::your_bucket_name"
      ]
    }
  ]
}

Tuesday, November 29, 2016

Amazon S3 grants permission to four types of users

Amazon S3 grants permission to four types of users, namely:
  1. Owner (account holder)
    Person who holds an Amazon S3 account is also known as the owner of the service. By default, the owner has full permission. The owner can create, access, and delete objects. He/She can also view and modify ACLs of each and every Bucket and its object(s).
  2. Amazon S3 Users (by adding Amazon.com email address or Canonical Id)
    If the owner wants to share or allow another Amazon S3 user to access his/her Bucket, then the owner should know the email address of the invitee. Email address only works if the invitee has registered his/her Amazon S3 account with that email address. You can also do this with Canonical ID instead of email address.
  3. Authenticated User (Sharing globally with all Amazon S3 Users)
    Anyone with a valid S3 account is a member of the "Authenticated Users" group. If Owner wants to share his/her Bucket globally with all Amazon S3 users, then he/she can give read permission to authenticated users to see the objects and can give write permission to update existing objects and upload new objects.
  4. Non Authenticated Users (All Users)
    If Owner wants to make his/her Bucket and objects public with all internet users, he/she needs to give the appropriate permissions to ALL USERS. Now any user will be able to access the object provided by the name of the Bucket.
NOTE: We strongly recommend that in most cases, you avoid setting Bucket ACL as "read all" permission. It will be true when you are using Bucket for web hosting or when using Public Distribution, S3 Website features.
Did you know?
Permission set for a Bucket does NOT automatically propagate to files stored in that Bucket.

"Read" permission at Bucket level does NOT mean that the authorized user can read all the files in that Bucket. If you have Read permission on a Bucket that you do not own, that means that you are authorized for "list Bucket" request on that Bucket, which essentially means that if you give "read" permission on a Bucket to everyone, then everyone can list the file names, their size and last modified date from that Bucket. In most cases, this is not a recommended option to give " read " permission to everyone.

Resource Link:
http://www.bucketexplorer.com/documentation/amazon-s3--access-control-list-acl-overview.html

Friday, November 25, 2016

Android In-App Billing v3 using ServiceConnection Class

In this tutorial, we are going to learn how to integrate android In-App Purchase using ServiceConnection class.
In-app purchase is a way to sell non-physical or digital goods or services in your app. If you are planning to sell physical goods, I will suggest you go through my tutorial on Android Pay or Android Paypal Integration Tutorial
Most of the tutorials online make use of the android source code example TrivialDrive Project but in this tutorial like the title suggested, we are going to make use of the ServiceConnection class.
The latest version of android in-app billing API is version 3 and this tutorial will focus on this version.

The Android Application Screenshot

 

In-App Billing Version 3 features

In-app Billing Version 3 provides the following features:
Your app sends requests through a streamlined API that allows users to easily request product details from Google Play and order in-app products. The API quickly restores products based on the user’s ownership.
The API synchronously propagates order information to the device on purchase completion.
All purchases are managed (that is, Google Play keeps track of the user’s ownership of in-app products). The user can’t own multiple copies of an in-app product; only one copy can be owned at any point in time.
Purchased products can be consumed. When consumed, the product reverts to the unowned state and can be purchased again from Google Play.
The API provides support for subscriptions.
Please use the comment box to answer questions related to problem you are facing with the code or suggestions on how to improve the tutorial.
This android in-app billing v3 tutorial has different sections to make it easy to understand.

Install or update Google Play Billing Library

We are going to check if we have Google Play Billing Library installed or if it needs to be updated.
In Android Studio, click the SDK Manager menu icon and the click on Launch Standalone SDK Manager.
In the Android SDK window, scroll down to the Extras category and select Google Play Billing Library. Install or update it is need be.


Prepare Store in your Google Play Developer Console

To test your android app for in-app billing, you need to log into your Google Play Developer Console. If you have not registered with Google as an app publisher, then you will first of all do that.
In the developer console, check if if have a Google Wallet attached to your account.

Create a new Google Wallet Merchant Account

To create a new Google Wallet Merchant Account, Go the Settings menu option inside your developer console page.
Then select Account details and scroll down to the bottom of the page.
Click on create merchant account button.
Follow the remaining instruction to finish up
When you are done, you will see a screen like the one shown below.



For the store list, we will complete the following sections.
1. Store Listing
2. Content Rating
3. Pricing and Distribution
4. APK – For the apk section, will add a signed apk when we create our project.


How to add In-App purchase Products

Go to the In-app product menu in the left hand side of the page
There are two type of in-app purchase products – Managed product and Subscription products
We will focus on the Managed Product
Click on Add new product button
Add the title and description for the product
Set a price for the product
When you are done, make sure you click on the inactive button and activate this product.
We have add the following products as shown in the screen shot below.


CREATE ANDROID APPLICATION

Lets start to soil our hands in code. Start up your IDE. For this tutorial, I am using the following tools and environment, feel free to use what works for you.
Windows 10
Android Studio
Sony Xperia ZL
Min SDK 14
Target SDK 23
To create a new android application project, follow the steps as stipulated below.
Go to File menu
Click on New menu
Click on Android Application
Enter Project name: AndroidInAppPurchase
Package: com.inducesmile.androidinapppurchase
Select Blank Activity
Name your activity : MainActivity
Keep other default selections
Continue to click on next button until Finish button is active, then click on Finish Button.

Steps to follow to Integrate In-app Billing library in our project

Update your AndroidManifest.xml file.
Create a ServiceConnection and bind it to IInAppBillingService.
Send In-app Billing requests from your application to IInAppBillingService.
Handle In-app Billing responses from Google Play.

Update AndroidManifest.xml

We are going to update the AndroidManifest.xml file by adding the following user permissions

<uses-permission android:name="com.android.vending.BILLING" />
 
The android vending billing permission handles all communication between your application and the Google Play server.
Open the Manifest.xml file and add the permissions

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.inducesmile.androidinapppurchase">
    <uses-permission android:name="com.android.vending.BILLING" />
    <uses-permission android:name="android.permission.INTERNET" />
    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:supportsRtl="true"
        android:theme="@style/AppTheme">
        <activity android:name=".MainActivity">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
        <activity android:name=".InAppPurchaseActivity" />
        <activity android:name=".PrivateContentActivity"> </activity>
    </application>
</manifest>

Adding the AIDL file to your project

IInAppBillingService.aidl is an Android Interface Definition Language (AIDL) file that defines the interface to the In-app Billing Version 3 service. You will use this interface to make billing requests by invoking IPC method calls.
To get the AIDL file:
Open the Android SDK Manager.
In the SDK Manager, expand the Extras section.
Select Google Play Billing Library.
Click Install packages to complete the download.
The IInAppBillingService.aidl file will be installed to /extras/google/play_billing/.

To add the AIDL to your project:

First, download the Google Play Billing Library to your Android project:
Select Tools > Android > SDK Manager.
Under Appearance & Behavior > System Settings > Android SDK, select the SDK Tools tab to
select and download Google Play Billing Library.
Next, copy the IInAppBillingService.aidl file to your project.
If you are using Android Studio:
Navigate to src/main in the Project tool window.
Select File > New > Directory and enter aidl in the New Directory window, then select OK.
Select File > New > Package and enter com.android.vending.billing in the New Package window, then select OK.
Using your operating system file explorer, navigate to /extras/google/play_billing/, copy the IInAppBillingService.aidl file, and paste it into the com.android.vending.billing package in your project.
If you are developing in a non-Android Studio environment: Create the following directory /src/com/android/vending/billing and copy the IInAppBillingService.aidl file into this directory. Put the AIDL file into your project and use the Gradle tool to build your project so that the IInAppBillingService.java file gets generated.
Build your application. You should see a generated file named IInAppBillingService.java in the /gen directory of your project.

Update Dependencies in build.gradle file

We are going to add some library dependencies in the app build.gradle file. The updated version of the app build.gradle file is below.

apply plugin: 'com.android.application'
android {
    compileSdkVersion 24
    buildToolsVersion "24.0.0"
    defaultConfig {
        applicationId "com.inducesmile.androidinapppurchase"
        minSdkVersion 14
        targetSdkVersion 24
        versionCode 2
        versionName "1.1"
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
}
dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    testCompile 'junit:junit:4.12'
    compile 'com.android.support:appcompat-v7:24.1.1'
    compile 'com.intuit.sdp:sdp-android:1.0.3'
    compile 'com.android.support:support-annotations:24.1.1'
    compile 'org.jetbrains:annotations-java5:15.0'
}


STRINGS.XML

We are going to update our project strings.xml file located in the values folder inside the res folder. Open the file and add the code below to it.

<resources>
    <string name="app_name">Android In-App Purchase</string>
    <string name="slogan">Using in-app purchase to make money</string>
    <string name="download_button">CLICK TO DOWNLOAD FILE</string>
    <string name="buy_gold">BUY GOLD FOR YOUR APP</string>
    <string name="in_app_support">In-app purchase is not supported by your device</string>
    <string name="main_content">Thanks for your purchase and welcome to the paid content section</string>
</resources>

COLORS.XML

Open the colors.xml file in the same location as the strings.xml file and add the code below to the file.

<?xml version="1.0" encoding="utf-8"?>
<resources>
    <color name="colorPrimary">#FF9800</color>
    <color name="colorPrimaryDark">#F57C00</color>
    <color name="colorPrimaryLight">#FFE0B2</color>
    <color name="colorAccent">#FF5722</color>
    <color name="colorPrimaryText">#212121</color>
    <color name="colorSecondaryText">#757575</color>
    <color name="colorIcons">#ffffff</color>
    <color name="colorDivider">#BDBDBD</color>
</resources>

MainActivity.java and activity_main.xml

When we created our project, Android Studio created a default MainActivity.java file and a corresponding activity_main.xml layout file located in the layout folder.
We will use the page as an intro splash page for our application.

activity_main.xml

open the activity_main.xml file and add the code below.

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:background="@color/colorPrimaryLight"
    tools:context="com.inducesmile.androidinapppurchase.MainActivity">
    <ImageView
        android:id="@+id/logo"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:contentDescription="@string/app_name"
        android:layout_centerVertical="true"
        android:src="@drawable/logos"
        android:layout_centerHorizontal="true"/>
    <TextView
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_centerHorizontal="true"
        android:textColor="@color/colorPrimaryDark"
        android:textStyle="italic"
        android:text="@string/slogan"
        android:layout_marginLeft="@dimen/_16sdp"
        android:layout_marginRight="@dimen/_16sdp"
        android:layout_marginTop="@dimen/_4sdp"
        android:layout_below="@+id/logo"/>
</RelativeLayout>

MainActivity class

The MainActivity class will instantiate a handle with a postDelay of three seconds before the next Activity page will load.

import android.content.Intent;
import android.content.pm.ActivityInfo;
import android.os.Handler;
import android.support.v7.app.ActionBar;
import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.view.Window;
import android.view.WindowManager;
public class MainActivity extends AppCompatActivity {
    private final int SPLASH_DISPLAY_LENGTH = 5000;
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        requestWindowFeature(Window.FEATURE_NO_TITLE);
        getWindow().setFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN, WindowManager.LayoutParams.FLAG_FULLSCREEN);
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_PORTRAIT);
        ActionBar actionBar = getSupportActionBar();
        if(null != actionBar){
            actionBar.hide();
        }
        new Handler().postDelayed(new Runnable(){
            @Override
            public void run(){
                Intent startActivityIntent = new Intent(MainActivity.this, InAppPurchaseActivity.class);
                startActivity(startActivityIntent);
                MainActivity.this.finish();
            }
        }, SPLASH_DISPLAY_LENGTH);
    }
}

InAppPurchaseActivity.java and activity_in_app_purchase.xml

This is where will offer our app users the opportunity to to make in-app purchase. In the layout file, we will give the user the opportunity to make purchase in different denominations.

activity_in_app_purchase.xml

open the layout file above and add the code below to it.
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:orientation="vertical"
    android:background="@color/colorPrimaryLight"
    tools:context="com.inducesmile.androidinapppurchase.InAppPurchaseActivity">
    <TextView
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_gravity="center_horizontal"
        android:text="@string/buy_gold"
        android:textSize="@dimen/_20sdp"
        android:layout_marginTop="@dimen/_60sdp"
        android:textColor="@color/colorPrimaryText"
        android:textStyle="bold"/>
    <LinearLayout
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:paddingTop="@dimen/_20sdp"
        android:gravity="center"
        android:orientation="horizontal">
        <LinearLayout
            android:layout_width="0dp"
            android:layout_height="wrap_content"
            android:layout_weight="1"
            android:gravity="center"
            android:orientation="vertical">
            <ImageView
                android:id="@+id/buy_one"
                android:layout_width="wrap_content"
                android:layout_height="wrap_content"
                android:layout_margin="@dimen/_4sdp"
                android:adjustViewBounds="true"
                android:src="@drawable/buyone"
                android:contentDescription="@string/app_name"/>
        </LinearLayout>
        <LinearLayout
            android:layout_width="0dp"
            android:layout_height="wrap_content"
            android:layout_weight="1"
            android:gravity="center"
            android:orientation="vertical">
            <ImageView
                android:id="@+id/buy_two"
                android:layout_width="wrap_content"
                android:layout_height="wrap_content"
                android:layout_margin="@dimen/_4sdp"
                android:src="@drawable/buytwo"
                android:adjustViewBounds="true"
                android:contentDescription="@string/app_name"/>
        </LinearLayout>
        <LinearLayout
            android:layout_width="0dp"
            android:layout_height="wrap_content"
            android:layout_weight="1"
            android:gravity="center"
            android:orientation="vertical">
            <ImageView
                android:id="@+id/buy_three"
                android:layout_width="wrap_content"
                android:layout_height="wrap_content"
                android:layout_margin="@dimen/_4sdp"
                android:src="@drawable/buythree"
                android:adjustViewBounds="true"
                android:contentDescription="@string/app_name"/>
        </LinearLayout>
    </LinearLayout>
</LinearLayout>

InAppPurchaseActivity.java

For the InAppPurchaseActivity class, we will get the imstances of the View controls that represent the buy buttons for all the three products we have added.
Before a user makes a purchase, we will check if billing is supported with the method below
public boolean isBillingSupported(){
    int response = 1;
    try {
        response = mService.isBillingSupported(3, getPackageName(), "inapp");
    } catch (RemoteException e) {
        e.printStackTrace();
    }
    if(response > 0){
        return false;
    }
    return true;
}
For the whole process of managing android in-app billing, we will create the following methods to execute the tasks mentioned.
1. consumePurchaseItem(String purchaseToken) – it is used to consume available purchase items
2. getAllUserPurchase() – get a list of all the purchases a user has made
3. purchaseItem(String sku) – used to purchase an item with the item id passed as parameter
4. itemPurchaseAvailability() – used to check available items to purchase
An instance of the ServiceConnection class is create and its service is bind to the Interface In-App Billing Service as shown below
ServiceConnection mServiceConn = new ServiceConnection() {
    @Override
    public void onServiceDisconnected(ComponentName name) {
        mService = null;
    }
    @Override
    public void onServiceConnected(ComponentName name, IBinder service) {
        mService = IInAppBillingService.Stub.asInterface(service);
        AvailablePurchaseAsyncTask mAsyncTask = new AvailablePurchaseAsyncTask(appPackageName);
        mAsyncTask.execute();
    }
};
Please note that getAllUserPurchase() and itemPurchaseAvailability() methods should be called in non UI Thread to avoid app crashing.
The complete code for the InAppPurchaseActivity class is as shown below.
import android.app.PendingIntent;
import android.content.ComponentName;
import android.content.Context;
import android.content.Intent;
import android.content.IntentSender;
import android.content.ServiceConnection;
import android.os.AsyncTask;
import android.os.Bundle;
import android.os.IBinder;
import android.os.RemoteException;
import android.support.v7.app.AppCompatActivity;
import android.view.View;
import android.widget.ImageView;
import com.android.vending.billing.IInAppBillingService;
import com.inducesmile.androidinapppurchase.entity.AvailablePurchase;
import com.inducesmile.androidinapppurchase.entity.UserPurchaseItems;
import com.inducesmile.androidinapppurchase.helpers.Helper;
import com.inducesmile.androidinapppurchase.session.CustomSharedPreference;
import org.json.JSONException;
import org.json.JSONObject;
import java.math.BigInteger;
import java.security.SecureRandom;
import java.util.ArrayList;
import java.util.List;
import java.util.Random;
public class InAppPurchaseActivity extends AppCompatActivity {
    private static final String TAG = InAppPurchaseActivity.class.getSimpleName();
    private IInAppBillingService mService;
    private CustomSharedPreference customSharedPreference;
    String[] productIds = new String[]{Helper.ITEM_ONE_ID, Helper.ITEM_TWO_ID, Helper.ITEM_THREE_ID};
    private ImageView buyOneButton, buyTwoButton, buyThreeButton;
    private static final char[] symbols = new char[36];
    static {
        for (int idx = 0; idx < 10; ++idx)
            symbols[idx] = (char) ('0' + idx);
        for (int idx = 10; idx < 36; ++idx)
            symbols[idx] = (char) ('a' + idx - 10);
    }
    private String appPackageName;
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_in_app_purchase);
        appPackageName = this.getPackageName();
        Intent serviceIntent = new Intent("com.android.vending.billing.InAppBillingService.BIND");
        serviceIntent.setPackage("com.android.vending");
        bindService(serviceIntent, mServiceConn, Context.BIND_AUTO_CREATE);
        customSharedPreference = new CustomSharedPreference(InAppPurchaseActivity.this);
        buyOneButton = (ImageView)findViewById(R.id.buy_one);
        buyOneButton.setVisibility(View.GONE);
        assert buyOneButton != null;
        buyOneButton.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View view) {
                if(!isBillingSupported()){
                    Helper.displayMessage(InAppPurchaseActivity.this, getString(R.string.in_app_support));
                    return;
                }
                purchaseItem(Helper.ITEM_ONE_ID);
            }
        });
        buyTwoButton = (ImageView)findViewById(R.id.buy_two);
        buyTwoButton.setVisibility(View.GONE);
        assert buyTwoButton != null;
        buyTwoButton.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View view) {
                if(!isBillingSupported()){
                    Helper.displayMessage(InAppPurchaseActivity.this, getString(R.string.in_app_support));
                    return;
                }
                purchaseItem(Helper.ITEM_TWO_ID);
            }
        });
        buyThreeButton = (ImageView)findViewById(R.id.buy_three);
        buyThreeButton.setVisibility(View.GONE);
        assert buyThreeButton != null;
        buyThreeButton.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View view) {
                if(!isBillingSupported()){
                    Helper.displayMessage(InAppPurchaseActivity.this, getString(R.string.in_app_support));
                    return;
                }
                purchaseItem(Helper.ITEM_THREE_ID);
            }
        });
    }
    ServiceConnection mServiceConn = new ServiceConnection() {
        @Override
        public void onServiceDisconnected(ComponentName name) {
            mService = null;
        }
        @Override
        public void onServiceConnected(ComponentName name, IBinder service) {
            mService = IInAppBillingService.Stub.asInterface(service);
            AvailablePurchaseAsyncTask mAsyncTask = new AvailablePurchaseAsyncTask(appPackageName);
            mAsyncTask.execute();
        }
    };
    private void purchaseItem(String sku){
        String generatedPayload = getPayLoad();
        customSharedPreference.setDeveloperPayLoad(generatedPayload);
        try {
            Bundle buyIntentBundle = mService.getBuyIntent(3, getPackageName(), sku, "inapp", generatedPayload);
            PendingIntent pendingIntent = buyIntentBundle.getParcelable("BUY_INTENT");
            try {
                startIntentSenderForResult(pendingIntent.getIntentSender(), Helper.RESPONSE_CODE, new Intent(), Integer.valueOf(0), Integer.valueOf(0), Integer.valueOf(0));
            } catch (IntentSender.SendIntentException e) {
                e.printStackTrace();
            }
        } catch (RemoteException e) {
            e.printStackTrace();
        }
    }
    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        if (requestCode == Helper.RESPONSE_CODE) {
            int responseCode = data.getIntExtra("RESPONSE_CODE", 0);
            String purchaseData = data.getStringExtra("INAPP_PURCHASE_DATA");
            String dataSignature = data.getStringExtra("INAPP_DATA_SIGNATURE");
            if (resultCode == RESULT_OK) {
                try {
                    JSONObject purchaseJsonObject = new JSONObject(purchaseData);
                    String sku = purchaseJsonObject.getString("productId");
                    String developerPayload = purchaseJsonObject.getString("developerPayload");
                    String purchaseToken = purchaseJsonObject.getString("purchaseToken");
                    //the developerPayload value is better stored in remote database but in this tutorial
                    //we will use a shared preference
                    for(int i = 0; i < productIds.length; i++){
                        if(productIds[i].equals(sku) && developerPayload.equals(customSharedPreference.getDeveloperPayload())){
                            customSharedPreference.setPurchaseToken(purchaseToken);
                            //access to private content
                            Intent contentIntent = new Intent(InAppPurchaseActivity.this, PrivateContentActivity.class);
                            startActivity(contentIntent);
                        }
                    }
                }
                catch (JSONException e) {
                    e.printStackTrace();
                }
            }
        }
    }
    private String getPayLoad(){
        RandomString randomString = new RandomString(36);
        String payload = randomString.nextString();
        return payload;
    }
    public class RandomString {
        private final Random random = new Random();
        private final char[] buf;
        public RandomString(int length) {
            if (length < 1)
                throw new IllegalArgumentException("length < 1: " + length);
            buf = new char[length];
        }
        public String nextString() {
            for (int idx = 0; idx < buf.length; ++idx)
                buf[idx] = symbols[random.nextInt(symbols.length)];
            return new String(buf);
        }
    }
    public final class SessionIdentifierGenerator {
        private SecureRandom random = new SecureRandom();
        public String nextSessionId() {
            return new BigInteger(130, random).toString(32);
        }
    }
    private class AvailablePurchaseAsyncTask extends AsyncTask<Void, Void, Bundle> {
        String packageName;
        public AvailablePurchaseAsyncTask(String packageName){
            this.packageName = packageName;
        }
        @Override
        protected Bundle doInBackground(Void... voids) {
            ArrayList<String> skuList = new ArrayList<String>();
            skuList.add(Helper.ITEM_ONE_ID);
            skuList.add(Helper.ITEM_TWO_ID);
            skuList.add(Helper.ITEM_THREE_ID);
            Bundle query = new Bundle();
            query.putStringArrayList(Helper.ITEM_ID_LIST, skuList);
            Bundle skuDetails = null;
            try {
                skuDetails = mService.getSkuDetails(3, packageName, "inapp", query);
            } catch (RemoteException e) {
                e.printStackTrace();
            }
            return skuDetails;
        }
        @Override
        protected void onPostExecute(Bundle skuDetails) {
            List<AvailablePurchase> canPurchase = new ArrayList<AvailablePurchase>();
            int response = skuDetails.getInt("RESPONSE_CODE");
            if (response == 0) {
                ArrayList<String> responseList = skuDetails.getStringArrayList("DETAILS_LIST");
                if(responseList != null){
                    for (String thisResponse : responseList) {
                        JSONObject object = null;
                        try {
                            object = new JSONObject(thisResponse);
                            String sku = object.getString("productId");
                            String price = object.getString("price");
                            canPurchase.add(new AvailablePurchase(sku, price));
                        } catch (JSONException e) {
                            e.printStackTrace();
                        }
                    }
                }
            }
            if(checkIfPurchaseIsAvailable(canPurchase, productIds[0])){
                buyOneButton.setVisibility(View.VISIBLE);
            }else{
                buyOneButton.setVisibility(View.GONE);
            }
            if(checkIfPurchaseIsAvailable(canPurchase, productIds[1])){
                buyTwoButton.setVisibility(View.VISIBLE);
            }else{
                buyTwoButton.setVisibility(View.GONE);
            }
            if(checkIfPurchaseIsAvailable(canPurchase, productIds[2])){
                buyThreeButton.setVisibility(View.VISIBLE);
            }else{
                buyThreeButton.setVisibility(View.GONE);
            }
        }
    }
    @org.jetbrains.annotations.Contract("null, _ -> false")
    private boolean checkIfPurchaseIsAvailable(List<AvailablePurchase> all, String productId){
        if(all == null){ return false;}
        for(int i = 0; i < all.size(); i++){
            if(all.get(i).getSku().equals(productId)){
                return true;
            }
        }
        return false;
    }
    public boolean isBillingSupported(){
        int response = 1;
        try {
            response = mService.isBillingSupported(3, getPackageName(), "inapp");
        } catch (RemoteException e) {
            e.printStackTrace();
        }
        if(response > 0){
            return false;
        }
        return true;
    }
    public void consumePurchaseItem(String purchaseToken){
        try {
            int response = mService.consumePurchase(3, getPackageName(), purchaseToken);
            if(response != 0){
                return;
            }
        } catch (RemoteException e) {
            e.printStackTrace();
        }
    }
    public Bundle getAllUserPurchase(){
        Bundle ownedItems = null;
        try {
            ownedItems = mService.getPurchases(3, getPackageName(), "inapp", null);
        } catch (RemoteException e) {
            e.printStackTrace();
        }
        return ownedItems;
    }
    public List<UserPurchaseItems> extractAllUserPurchase(Bundle ownedItems){
        List<UserPurchaseItems> mUserItems = new ArrayList<UserPurchaseItems>();
        int response = ownedItems.getInt("RESPONSE_CODE");
        if (response == 0) {
            ArrayList<String> ownedSkus = ownedItems.getStringArrayList("INAPP_PURCHASE_ITEM_LIST");
            ArrayList<String>  purchaseDataList = ownedItems.getStringArrayList("INAPP_PURCHASE_DATA_LIST");
            ArrayList<String>  signatureList = ownedItems.getStringArrayList("INAPP_DATA_SIGNATURE_LIST");
            String continuationToken = ownedItems.getString("INAPP_CONTINUATION_TOKEN");
            if(purchaseDataList != null){
                for (int i = 0; i < purchaseDataList.size(); ++i) {
                    String purchaseData = purchaseDataList.get(i);
                    assert signatureList != null;
                    String signature = signatureList.get(i);
                    assert ownedSkus != null;
                    String sku = ownedSkus.get(i);
                    UserPurchaseItems allItems = new UserPurchaseItems(sku, purchaseData, signature);
                    mUserItems.add(allItems);
                }
            }
        }
        return mUserItems;
    }
    @Override
    public void onDestroy() {
        super.onDestroy();
        if (mService != null) {
            unbindService(mServiceConn);
        }
    }
}

PrivateContentActivity and activity_private_content.xml

If the purchase is successful, the app will take the user to the Private Content page which is not available for unpaid users.
import android.os.Bundle;
import android.support.v7.app.AppCompatActivity;

public class PrivateContentActivity extends AppCompatActivity{
    private CustomSharedPreference customSharedPreference;
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_private_content);
        
        //You can set the condition for consumption of the purchase or it can be one time fee
        //it depends on your application requirement.
        // but if you to consume a purchase, call the method below
        //String token = customSharedPreference.getPurchaseToken();
        // You can call consume purchase
        //consumePurchaseItem(token)
    }
}

Create Entity Package Directory

We will create a new package folder and name it entity. Create the files below and add the contents respectively.

AvailablePurchase.java

public class AvailablePurchase {
    private String sku;
    private String price;
    public AvailablePurchase(String sku, String price) {
        this.sku = sku;
        this.price = price;
    }
    public String getSku() {
        return sku;
    }
    public String getPrice() {
        return price;
    }
}

UserPurchaseItems.java

public class UserPurchaseItems {
    private String sku;
    private String purchasedata;
    private String signature;
    public UserPurchaseItems(String sku, String purchasedata, String signature) {
        this.sku = sku;
        this.purchasedata = purchasedata;
        this.signature = signature;
    }
    public String getSku() {
        return sku;
    }
    public String getPurchasedata() {
        return purchasedata;
    }
    public String getSignature() {
        return signature;
    }
}

Create Helper Package Directory

We will create a new package folder and name it helpers. Inside the package, create a new java file and name it Helper.java.
Add the code below to it.

Helper.java

import android.content.Context;
import android.widget.Toast;
public class Helper {
    public static final String ITEM_ID_LIST = "ITEM_ID_LIST";
    public static final String ITEM_ONE_ID = "productone";
    public static final String ITEM_TWO_ID = "producttwo";
    public static final String ITEM_THREE_ID = "productthree";
    public static final int RESPONSE_CODE = 1001;
    public static final String SHARED_PREF = "shared_pref";
    public static final String DEVELOPER_PAYLOAD = "developer_payload";
    public static final String PURCHASE_TOKEN = "purchase_token";
    public static void displayMessage(Context context, String message){
        Toast.makeText(context.getApplicationContext(), message, Toast.LENGTH_LONG).show();
    }
}

Testing In-App Billing Purchase

To test you are in-app purchase, create a Google+ account and add the users that will test the app in your group or community. Do not use your main Google account to test your In-App purchase.

Errors You might encounter during In-App purchase testing

the item you requested is not available for purchase
Solution – According to AndreiBogdan in Stackoverflow,
Here are some things to check:
1. You’ve created an apk and you’ve published it to the GooglePlay Dashboard in Alpha or Beta.
2. The app in the GooglePlay Dashboard is NOT in Draft mode, but in Published (you’ll need to make all the small circles with the check icon in them on the left side of the screen green before being able to publish).
3. You’ve set another test account than the one that’s “attached” to the GooglePlay Dashboard. You can do that by creating a Google+ group, add your test account to that group and specify the Google+ group in the GooglePlay Dashboard.
4. The apk that you’re using to test the purchase has the same version code, version name, and most importantly it’s signed with the same keystore as the apk that you’ve published in the store.
5. You wait a couple of hours between when you change something in the dashboard in order for the changes to propagate. It takes a couple of hours to do so.
6. Make sure the sku value is a valid sku value (compare it with the one you’ve entered in the GP Dashboard).
7. You try to purchase an already purchased item. Get the purchased items and display them in the log to see if so. If so, then consume that product or refund the money to your test account(you’ll need to wait for the refund to propagate. It takes a couple of hours.)
8. Make sure the Inapps are Active !
9. Make sure you’re signedIn into google (in your browser) with the test account and you open this link (marked with the red) and you approve to become a tester !!!!

This brings us to the end of this tutorial. I hope that you have learn something. Run your app and see for yourself.
You can download the code for this tutorial below. If you are having hard time downloading the tutorial, kindly contact me.
Remember to subscribe with your email address so that you will be among the first to receive my new android blog post once it is published.


N.B: All data's are copy pasted from this link for study purpose: https://inducesmile.com/android/android-in-app-billing-v3-purchase-using-serviceconnection-example-tutorial/#