Crawling mobile app stores with F#

Some time ago I needed a way to programatically search the Apple AppStore and Google Play Store to get some info about apps for a project. I decided to write an F# script for that task and later added support for Windows Phone Store.

Types

I wanted the script to be easily usable from outside of F# so first I created a type for the app.

I also needed a helper function to download data from the web. I used a classic WebClient with user agent set to Chrome, because the Windows Phone Store API requires a user agent header

Apple AppStore

Searching in Apple AppStore is easy, because Apple provides a simple search API that returns JSON. Thanks to the JsonProvider from FSharp.Data, the code is really simple

Just downloading the JSON transforming the data to our type.

Google Play Store

Searching in Google Play Store is a bit more of a challenge. There is no API I could found so I had to parse the html returned from the web version of Google Play Store. The web can change at any time so I need to be aware of this fact and fix the method if it happens.

Another problem with parsing the web is that there is no way to return search result for a specific country like for Apple AppStore. The web always shows results according to your IP address.

Windows Phone Store

There is no official API for searching the Windows Phone Store but I was given a tip obtained after sniffing the traffic from a Windows Phone, so I did not have to parse the web. The code is as simple as for Apple AppStore thanks to XmlProvider.

Conclusion

Using F# is fun and processing data is really simple thanks to type provides. You can find it whole code at https://github.com/igorkulman/AppStoreCrawler.

0  

Building Windows Phone apps with FAKE

FAKE is a build automation system with capabilities which are similar to make and rake. It is using an easy domain-specific language (DSL) so that you can start using it without learning F#. If you need more than the default functionality you can either write F# or simply reference .NET assemblies.

I have been using FAKE for quite some time now on some fairly complex projects for not only building but also running tests and creating and pushing Nuget packages and I really like. I decided to add FAKE build scripts to my Windows Phone apps to make the process of generating a XAP file for the Windows Phone Store easier.

The FAKE script I use can by used with any Windows Phone app, it will build all the projects and copy the XAP file to a release directory.

To get the script started, you need a batch file

and Nuget.exe in tools\NuGet directory.

2  

How to get rid of the strange line under systray in Windows Phone 8

If you create an Windows Phone 8 app and test it only on WVGA and 720p devices or emulators, you may be surprised how you app looks on a WXGA device (or emulator).

I have not been able the reason why this happens but the solution is quite simple. Set your page’s top border to -1.

Doing this in XAML for every page is not very convenient, a better solution would be to set the negative top margin on the whole application frame

If you use Caliburn.Micro, you need to override the CreatePhoneApplicationFrame in the Bootstrapper instead

1  

Getting contacts on Windows Phone the async/await way

Getting contacts info on Windows Phone means using a callback based API provided by the Windows Phone SDK

I really dislike all the callback-based API so I was looking for a way to convert it to an async/await based API. And it is quite easy to do so

You can find this implementation in my Kulman.WP8 library (also available on Nuget)

0  

Testing in-app purchases in Windows Phone

Windows Phone Store does not offer developers any sandbox to test in-app purchases in their apps, like stores on other platforms do. If you want test in-app purchases in your Windows Phone apps, you need to use other options.

To make in-app purchases implementation easier, I created a simple Windows Phone Store service interface in my Kulman.WP8 library (also available on Nuget)

Private beta

If you publish your app as a private beta and add the in-app products, you can test them. All the in-app purchases are always free in this scenario. The disadvantage is that you cannot debug anything, it either works or it does not (if you use my implementation from Kulman.WP8, it should :).

Real app testing

If your app is already in the Windows Phone Store, create an in-app product and try to buy it from your app run from Visual Studio, you may be wondering, why you are getting an error. The problem is that the app run from Visual Studio has a different app id from the app in Windows Phone Store. If you change the app id in the manifest to the app id state in Windows Phone Store (the details view), you will be able to make a real in-app purchase.

Mocking library

To mock in-app purchases you can use the Mock In-App Purchase Library. To make testing easier, implement my interface using this library

And setup the products you want to use at your app startup

The advantage of this approach is that you just switch the IWindowsStoreService implementation between then mock on and the real one, depending on the situation. For example (Caliburn.Micro Bootstrapper)

0  

Pock8: beautiful Pocket client for Windows Phone

Recently I have released a new Windows Phone app called Pock8. Pock8 is a beautiful Pocket (formerly Read It Later) client for Windows Phone. The design of the app was created by Jan Marek who worked with my on the Shopping List Simple app.

Pock8

Our goal was to create a Pocket client that would be really easy to use and that would have a nice and simple design. I think we have achieved our goal. The main features of the app are

  • clean and simple design
  • optimized article view for comfortable reading
  • article cache for offline reading
  • dark theme for OLED screens or reading in the dark
  • support for receiving articles from other apps like WPCentral
  • listening to articles aloud
  • sharing articles

In version 1.2 we really concentrated on user feedback and added many features that our users requested. We love and users and want to make them happy.

The app is free to use with one limitation, all the list display just 5 articles at any time. You can use an in-app purchase to get rid of the limitation and support further development of the app.

For more info about Pock8, follow the app on Twitter at @pock8app.

Pock8

0  

Free services to help you develop mobile apps

If you develop mobile or any other kinds of apps, you need to have your source code versioned and stored in a safe place, cooperate with other people, track information about the usage of your apps and track all the errors that may occur. There are many free services that may help you with those tasks. In this article I will give you a list of the ones I use when developing my apps.

Bitbucket: source control and issue tracker

I use Git as my source control system and all my private repos are stored at Bitbucket (public at GitHub). Bitbucket offers unlimited number of private repos and cooperation with up to 5 other people for free.

For each private repo you can create a private issue tracker also for free, so all your code and issues are at the same place. The main advantage is that source control and issue tracker are connected, so you can comment or close issues directly from Git commits.

Flurry: mobile analytics

I use Flurry Analytics to track the usage of my apps. Flurry supports Android, iOS, Windows Phone 8 and Blackberry. The tool is completely free and allows you to track events like app runs, unique users, page views or custom events with parameters. Tracking custom events with parameters is really handy, for example thanks to it I know how many users use which color theme in my Shopping List Simple app.

BugSense: bug tracking

Flurry also offers bug tracking, it is no very well done and you cannot get much use of it. To track application crashes and caught exceptions I use BugSense. BugSense supports Android, iOS, HTML5, Windows Phone and Windows 8 apps.

Each logged error (exception) provides a complete stracktrace, number of occurrences and detailed info about each instance, like app version, phone model, country, etc. BugSense can be connected to GitHub, Bitbucket is no supported. Maybe in the future.

BugSense is a really useful service that can help you easily identify problems in your apps. Free version offers you 7 days bug info retention for up to 500 different errors for unlimited amount of apps.

0  

C# scripting console for ASP.NET MVC application

In a recent larger ASP.NET MVC project running in Windows Azure I needed to provide the advanced users a way to execute custom scripts directly through the webbrowser. A kind of a scripting console where users can create and execute their own scripts, that interact with the project and automate some tasks (that would otherwise require too many clicks).

Choosing the language

I thought about using Lua but I needed an easy way to integrate with some classes (repositories) used in the project. I decided to use C# as the scripting language and implement the scripting console using Roslyn. Roslyn is a really neat project and I recommend taking a look at it, if you do not already know it.

I extracted the main idea of my implementation of the scripting console and posted it on GitHub. I will walk you through it in this blog post.

Basic implementation

My basic solution consists of a Textarea the user writes the script to, the script then gets executed by Roslyn and the final result is shown. I also included a basic Repository class to show, that the scripts can use classes from the project. In the real implementation I have a logger class showing the ongoing logs and results but as I said, the Github repo is just an extracted basic idea of the scripting console.

To execute a C# script in your ASP.NET MVC (or any other) application, you first need to install the Roslyn package and its depedencies

Let’s suppose you have the C# script you want to execute in a string variable called command obtained from the mentioned Textarea. First, you need to create the scripting engine and a session

There are ScriptEngine classes, one for C# and one for VB.NET so choose the one for C#. You can use any class as the context for the scripts. In the sample, I have a ScriptingContext class containing one public property of a type DataRepository. If you use a class as the context for the scripts, you can access all its properties (Repository of type DataRepository in my case) in the scripts.

To use a class as the context for the scripts, just pass it to the Roslyn session

If you want to use more than the core C# libraries, you need to do some referencing and importing. If you are using a class as the context for the scripts, you need to reference its assembly. Suppose you want to use classes and methods from System.Xml and System.Xml.Linq in your scripts. First, you need to add references to the session

and then import them to the session

Everything is set up now, so just execute the script and get the result

The result of the script is the result of the last expression of the script so I recommend using a logger class in the context if you want to get more information about the execution of your script.

Conclusion

Thanks to Roslyn, I was able to create a really usable scripting console for the project and allow users to automate many tasks using C# scripts. This article and the demo project describe just the basics, there are many ways to make the experience better. I for example use the CodeMirror editor to provide C# syntax highlighting, to allow users to upload ZIP files they can process in their scripts, etc.

Source code: WebConsole

0  

Creating a Search box with Reactive Extensions and MVVM

Having a Search box in a Windows Phone app is a common use-case but it is only rarely done right. A good Search box does not have a Search button associated with it that the users have to click when they are finished typing and want to start the search. A good Search box starts the search immediately when the user stops typing (for a certain period of time).

You can implement this functionality with some nasty code using a DispatcherTimer, or you can use Reactive Extensions. You can use Reactive Extensions directly on a TextBox representing the Search box, but if you use MVVM (and you should) you need to attach to the property associated with the Search box instead.

Suppose the ViewModel contains a string property called SearchTerm (with two-way binding to the Search box) and an event called PropertyChanged (used for the INotifyPropertyChanged implementation). We need to observe the changes of the SearchTerm property. There is no way to do it directly, we need to observe the PropertyChanged event instead.

We need to create an observable from the PropertyChanged event, subscribe to it filtering only the PropertyChanged calls regarding the SearchTerm property, throttling for 0.5 seconds

Using the throttle operator we will get exactly the desired behaviour of executing the Search method 0.5 seconds after the user stopped typing.

However, there is a small problem. The SearchTerm property changes (and fires the PropertyChanged event) only when the Search box loses focus. We need to make the SearchTerm property change after each letter typed instead. In WPF that would be trivial using UpdateTrigger, but Windows Phone does not support UpdateTrigger.

We need to create a custom binding utility

and apply it to the Search box

0  

Using the DebuggerDisplay attribute for better debugging experience

When debugging a C# program in Visual Studio, I tend to always hover over the variables to glance at their values and structure instead of explicitly writing their names into the watch window. If I want to explore say a collection, I need to unfold each of the items using the + button to get an idea about the data:

This is not very comfortable, so thankfully, there is a way to make this experience better, using the DebuggerDisplay attribute. This attribute can be applied to any class (and struct, enum, property, field, delegate, assembly) and allows you to define the information about the class you wanto to see in the debugger.

Give the attribute a string to be displayed and in this string you can reference any data from the class:

When you hover over the data now, you will see a nice readable “labels”:

debug2

If you do not want to add the DebuggerDisplay attribute to all your classes manually, you can use the Visualize addin for Fody. Visualize will add the DebuggerDisplay atribute to all your classes, using all the classe’s properties in the “label”.

Warning: showing this “labels” takes some time, so do not reference to many data fields.

1