Chromecast Functionality?

[ericbazinga]

With the rise of the Chromecast dongle (and the 100's of apps compatible with it), I thought that it would be cool to be able to use a TV as a second screen in DraStic- You could use the TV to show both screens or just the top screen and the phone as the touchscreen.

[TkSilver]

The problem with chromecast and miracast is latency. If you want a visual representation of this screen/control lag try this.

Get a stopwatch program, the more accurate the better, a tv with chromecast and a camera helps too.
In chromecast cast your screen and start the stopwatch app.
Place the phone where you can get a picture of both screens at the same time.

The time difference is the lag. Screen lag like this would be ok in games that do no require you to be accurate at all. So games like the professor layton series, phoenix wright series, and turn based strategy games would be mostly playable with the lag this will introduce. Any game that needs real time reaction speeds you would be handycapping yourself by generally .1 to .5 seconds and in games like mario that would be frustrating if no fatal.

All of these issues are not even including the extra overhead that casting live content causes.

[snoweyes]

I already do this in using miracast. It's best for in game maps and things that don't need much lag, ala Mario Kart tracks. Tracks on the TV, gaming screen on my tablet. It's great!

[nintynuts]

I actually tried this already with Pokémon, and it works OK. The only issues are the previously mentioned latency (although Pokémon is turn based, so it's ok but running and cycling are a pain), but also that you cannot have the top screen ONLY on the TV, you can only copy it, and it streams it at your full tablet/phone screen resolution (which is a waste as it's only running at 2-4 times the DS's resolution) and I think it's this bandwidth problem that gives me crackly audio, which ruins the whole experience.

To do it, cast your entire screen from the chromecast app, then in the drastic video settings, set the external display screen to top or bottom screen, and adjust your screen layout to one big, one small where the bigger screen is the one not on the TV.

It would be really nice if drastic had a chromecast app, as the data transmitted could be optimized to improve performance.

If we had texture upscaling using xBR, it would be incredible to play Pokémon on a big TV.

[ericbazinga]

I actually tried this already with Pokémon, and it works OK. The only issues are the previously mentioned latency (although Pokémon is turn based, so it's ok but running and cycling are a pain), but also that you cannot have the top screen ONLY on the TV, you can only copy it, and it streams it at your full tablet/phone screen resolution (which is a waste as it's only running at 2-4 times the DS's resolution) and I think it's this bandwidth problem that gives me crackly audio, which ruins the whole experience.

To do it, cast your entire screen from the chromecast app, then in the drastic video settings, set the external display screen to top or bottom screen, and adjust your screen layout to one big, one small where the bigger screen is the one not on the TV.

It would be really nice if drastic had a chromecast app, as the data transmitted could be optimized to improve performance.

If we had texture upscaling using xBR, it would be incredible to play Pokémon on a big TV.

We'd have to talk to Lordus about it... (hint hint)

[TkSilver]

I actually tried this already with Pokémon, and it works OK. The only issues are the previously mentioned latency (although Pokémon is turn based, so it's ok but running and cycling are a pain), but also that you cannot have the top screen ONLY on the TV, you can only copy it, and it streams it at your full tablet/phone screen resolution (which is a waste as it's only running at 2-4 times the DS's resolution) and I think it's this bandwidth problem that gives me crackly audio, which ruins the whole experience.

To do it, cast your entire screen from the chromecast app, then in the drastic video settings, set the external display screen to top or bottom screen, and adjust your screen layout to one big, one small where the bigger screen is the one not on the TV.

It would be really nice if drastic had a chromecast app, as the data transmitted could be optimized to improve performance.

If we had texture upscaling using xBR, it would be incredible to play Pokémon on a big TV.

We'd have to talk to Lordus about it... (hint hint)

If you cast your screen using chromecast then it is your device doing all the processing of the "video" content and chromecast just "playing" it. The cracklyness. Is partly from the underpowered hardware that chromecast uses because it is designed to stream video/audio/picture content and that is about it and your device effectively having to screen capture and stream that to the chromecast dongle. Optimization of a chromecast app means running a stripped down version of your app that is temp loaded on the 2GB flash it has and run it on the arm v5/v6 processor with 512MB of ram. There is a default program setup to handle very certain video streams it is pointed at (mp4 web content) and the rest is loaded by the device when it first connects to chromecast.

[nintynuts]

If you cast your screen using chromecast then it is your device doing all the processing of the "video" content and chromecast just "playing" it. The cracklyness. Is partly from the underpowered hardware that chromecast uses because it is designed to stream video/audio/picture content and that is about it and your device effectively having to screen capture and stream that to the chromecast dongle. Optimization of a chromecast app means running a stripped down version of your app that is temp loaded on the 2GB flash it has and run it on the arm v5/v6 processor with 512MB of ram. There is a default program setup to handle very certain video streams it is pointed at (mp4 web content) and the rest is loaded by the device when it first connects to chromecast.

I was aware of this, however the 'stripped down' app isn't necessarily accurate. As the app is downloaded from a URI when you connect to the chromecast, the app doesn't have to do what the app that launched it does, they just have to understand each other. What I was suggesting was that either a custom app is written to handle the DS content better (a lot of effort or possibly not worthwhile) or simply use an mp4 video decoding app and send over the video stream from the android device at the correct resolution, to then be scaled by the chromecast to fit the screen.

Maybe it's possible the whole screen GPU filter could be applied by the chromecast itself rather than the android device, which would mean quartering the transmitted resolution (at least).

Also, the chromecast can decode 5.1 AC3 audio without stuttering so stereo AAC (which I would guess is the native casting format) should be a doddle.
I think the performance issues come from the fact you're capturing your entire android screen and all audio channels, and compressing it on the GPU, rather than just compressing the audio and render target from the drastic app. I don't know at what point drastic hooks in to the screen capture and swaps it out for either just top or bottom screen, but after that happens I would imagine it's much the same.

[ericbazinga]

What I was suggesting was that either a custom app is written to handle the DS content better (a lot of effort or possibly not worthwhile) or simply use an mp4 video decoding app and send over the video stream from the android device at the correct resolution, to then be scaled by the chromecast to fit the screen.

The MP4 idea isn't bad, but I think we should look into other possible types of media formats and which has the least lag (because less lag equals more playable games).

[nintynuts]

The chromecast is engineered to decode mp4 (specifically h264) and VP8 (which is slower to encode and lower quality), so doing any other compression format would be worse. The question is what resolution to send the video in (the smaller the better) and whether it's possible to do some smarter up-scale filtering on the chromecast end (colour distortion from compression might mean this looks bad).

[ericbazinga]

The question is what resolution to send the video in (the smaller the better) and whether it's possible to do some smarter up-scale filtering on the chromecast end (colour distortion from compression might mean this looks bad).

Well, I suggest we take a DraStic screenshot and run it through downgrading software multiple times, and then find the variant that is small but not so small that the resulting image looks crappy.