Thanks for posting @hsiaoer and looking forward to your ML classification follow-up post. I've been focused on similar research and read with interest the ARM DynamIQ ML focus [0] as I believe this will open up ML edge processing on wearables, although hard to beat what Johny Srouji and team are doing.
WatchOS 4 seems to support CoreML [1], do you plan to do inferencing on the device eventually? If so, cool!
Yes I would love to get the CoreML model onto the watch itself. I think the interesting part will be tweaking the model and exploring how we can build it with space, battery consumption, and resource utilization in mind. The nice thing is that we can possibly deploy 2 models (one to the phone and one on the watch): the one on the phone runs when the watch is nearby and is more robust. The other runs directly on the watch itself with some tradeoffs). We might discover that we need better hardware on the watch but that would be interesting to explore too.
> Unless you’re extremely flexible, changes are that your palms won’t be facing away from your body (if you’ve put the watch on correctly).
Keep in mind that some users wear Apple Watch so that the screen face is on the inside of their wrist, rather than the outside. To a naïve interpretation this would appear as "palms facing away".
The orientation settings let you choose left or right wrist and, separately, left or right facing Digital Crown, which means you can turn the watch "upside down" and wear it that way. But it doesn't have a setting for wearing it on the inside of the wrist.
What if I use the watch at my right wrist and set is as left wrist?
Edit for clarity:
this is to make screen facing my inside and watchOS knows about this.
You can also consider using creating a model using the Activity Classifier toolkit in Turi Create which automatically exports in CoreML format for use in. your watchOS app
Awesome. Been meaning to play around with Turi Create. Seems pretty easy to get started with. One of my colleagues wrote a post on building the hotdog/not hotdog app with it in an afternoon. https://heartbeat.fritz.ai/building-not-hotdog-with-turi-cre...
I know it's just for educational use (thanks!), but fun to think through what it would take to make this into a fully functional app.
Exercises that don't involve arm movement would be impossible to classify. An extra sensor on the leg might be enough to a) handle cases where just leg movement is involved and b) capture body position for static exercises like planks and wall sits.
Yeah that is interesting and something that I wondered about while I wrote this post. There's only so many exercises you can model with a watch on your left wrist (at least with the accelerometer and gyroscope data). Another example that's been on the back of my mind was if I tried to go a step beyond classification to count reps for an exercise like single arm raises, would I be able to tell when I do a rep with my right arm? Definitely something I want to go a little deeper on in subsequent posts to find the limit of what we can and cannot classify. Like you said, I reckon static exercises would be pretty difficult and there would be a pretty high miss on those unless we had access to additional info.
check out Gymatic featured in Apple Watch app store. Exercise classification + rep counting. I am the developer and we already have a lot of active users using the solution in Gym.
I’ve just bought a new 3 series and it really bothers me how neuterd most of the apps feel. I have what’s equivalent to a desktop PC from 2005 on my wrist yet there is no web browser, decent games, or anything else actually pushing the hardware. The built in email client wont even render html or images. Google has dropped all app support but News. I’m not sure why people aren’t doing more with this platform, as the hardware specs are amazing.
>I’ve just bought a new 3 series and it really bothers me how neuterd most of the apps feel. I have what’s equivalent to a desktop PC from 2005 on my wrist yet there is no web browser, decent games, or anything else actually pushing the hardware.
Because you'd e.g. be web browsing on a watch if it did have a web browser?
Besides, even with the "neutered apps" and only cursory use of them, the battery barely lasts a full day. Imagine with what you're suggesting...
I get the point of what you're saying but, FYI, my Series 3 watch lasts 3 full days on a single charge. I typically wear it all day and then to bed for sleep tracking and then charge it for the hour or so that it takes me to shower, get ready, make breakfast, etc.
With a series 3, even the cellular version, the battery will easily last 2 days for most people. Even with heavy use I usually have over 50% battery left at the end of the day...
Curious: I just got a series 3 last week and use it as my sole phone (hate smartphone form factor, flip phone user since forever). All I use it for is heart rate, phone, messaging, and music (very happy with it).
I've had to charge my series 3 every night. I guess not having it paired with a phone somehow uses more battery power? Technically, I reset my wife's phone with my sim+apple id, paired, reset back to wife's backup and sim. So it has no real phone to pair with, though I guess maybe that causes it to use cellular a lot more than most.
I think part of it was that until the series 3, you still needed the phone nearby for wifi connectivity so developers were less interested in building a watch-specific app. Most of the ones out there were just widgets or notification apps that played nicely with the phone's.
As each series improves and Apple opens up new libraries so your watch becomes more and more independent of the phone, I believe we'll get more watch developers to up their game.
You don’t actually need the phone nearby with the previous watches either, they will connect directly to WiFi if they need to - they just prefer to use the phone for battery saving.
I was kinda surprised the first time I got an iMessage on my series 2 when my phone was in airplane mode!
I didn't get a chance to profile the app for battery consumption since my recordings were only 5-10s each but I'd assume that while the watch is recording the battery life would be significantly less if you had it on for an extended period of time.
Thanks for posting @hsiaoer and looking forward to your ML classification follow-up post. I've been focused on similar research and read with interest the ARM DynamIQ ML focus [0] as I believe this will open up ML edge processing on wearables, although hard to beat what Johny Srouji and team are doing.
WatchOS 4 seems to support CoreML [1], do you plan to do inferencing on the device eventually? If so, cool!
[0] https://community.arm.com/processors/b/blog/posts/arm-dynami...
[1] https://developer.apple.com/library/content/releasenotes/Gen...
Thanks for the link!
Yes I would love to get the CoreML model onto the watch itself. I think the interesting part will be tweaking the model and exploring how we can build it with space, battery consumption, and resource utilization in mind. The nice thing is that we can possibly deploy 2 models (one to the phone and one on the watch): the one on the phone runs when the watch is nearby and is more robust. The other runs directly on the watch itself with some tradeoffs). We might discover that we need better hardware on the watch but that would be interesting to explore too.
> Unless you’re extremely flexible, changes are that your palms won’t be facing away from your body (if you’ve put the watch on correctly).
Keep in mind that some users wear Apple Watch so that the screen face is on the inside of their wrist, rather than the outside. To a naïve interpretation this would appear as "palms facing away".
So many features are broken in this orientation that I doubt anyone would be surprised by this.
iirc, you can set the orientation in the Watch settings. The OS and software should adjust.
The orientation settings let you choose left or right wrist and, separately, left or right facing Digital Crown, which means you can turn the watch "upside down" and wear it that way. But it doesn't have a setting for wearing it on the inside of the wrist.
What if I use the watch at my right wrist and set is as left wrist? Edit for clarity: this is to make screen facing my inside and watchOS knows about this.
You can also consider using creating a model using the Activity Classifier toolkit in Turi Create which automatically exports in CoreML format for use in. your watchOS app
https://github.com/apple/turicreate/tree/master/userguide/ac...
Awesome. Been meaning to play around with Turi Create. Seems pretty easy to get started with. One of my colleagues wrote a post on building the hotdog/not hotdog app with it in an afternoon. https://heartbeat.fritz.ai/building-not-hotdog-with-turi-cre...
I know it's just for educational use (thanks!), but fun to think through what it would take to make this into a fully functional app.
Exercises that don't involve arm movement would be impossible to classify. An extra sensor on the leg might be enough to a) handle cases where just leg movement is involved and b) capture body position for static exercises like planks and wall sits.
Yeah that is interesting and something that I wondered about while I wrote this post. There's only so many exercises you can model with a watch on your left wrist (at least with the accelerometer and gyroscope data). Another example that's been on the back of my mind was if I tried to go a step beyond classification to count reps for an exercise like single arm raises, would I be able to tell when I do a rep with my right arm? Definitely something I want to go a little deeper on in subsequent posts to find the limit of what we can and cannot classify. Like you said, I reckon static exercises would be pretty difficult and there would be a pretty high miss on those unless we had access to additional info.
check out Gymatic featured in Apple Watch app store. Exercise classification + rep counting. I am the developer and we already have a lot of active users using the solution in Gym.
I'll take the bait. Great job on the app - seems like it really works. Also answers most questions about how this would be feasible
* How to track workouts where arms don't move? Add a second sensor, or create artificial arm movement.
- Want to track Leg Press? Strap phone to your leg. Or keep hands on your leg to move along.
* How to distinguish between similar workouts/add custom workouts? Custom classification per user means a smaller pool of workouts to distinguish.
- It takes just 3 seconds to train any new exercise
Looks neat! I'll try it out!
I’ve just bought a new 3 series and it really bothers me how neuterd most of the apps feel. I have what’s equivalent to a desktop PC from 2005 on my wrist yet there is no web browser, decent games, or anything else actually pushing the hardware. The built in email client wont even render html or images. Google has dropped all app support but News. I’m not sure why people aren’t doing more with this platform, as the hardware specs are amazing.
>I’ve just bought a new 3 series and it really bothers me how neuterd most of the apps feel. I have what’s equivalent to a desktop PC from 2005 on my wrist yet there is no web browser, decent games, or anything else actually pushing the hardware.
Because you'd e.g. be web browsing on a watch if it did have a web browser?
Besides, even with the "neutered apps" and only cursory use of them, the battery barely lasts a full day. Imagine with what you're suggesting...
I get the point of what you're saying but, FYI, my Series 3 watch lasts 3 full days on a single charge. I typically wear it all day and then to bed for sleep tracking and then charge it for the hour or so that it takes me to shower, get ready, make breakfast, etc.
With a series 3, even the cellular version, the battery will easily last 2 days for most people. Even with heavy use I usually have over 50% battery left at the end of the day...
Curious: I just got a series 3 last week and use it as my sole phone (hate smartphone form factor, flip phone user since forever). All I use it for is heart rate, phone, messaging, and music (very happy with it).
I've had to charge my series 3 every night. I guess not having it paired with a phone somehow uses more battery power? Technically, I reset my wife's phone with my sim+apple id, paired, reset back to wife's backup and sim. So it has no real phone to pair with, though I guess maybe that causes it to use cellular a lot more than most.
I'd imagine the constraints are not peak processing power but energy consumption and to some extent UI.
I think part of it was that until the series 3, you still needed the phone nearby for wifi connectivity so developers were less interested in building a watch-specific app. Most of the ones out there were just widgets or notification apps that played nicely with the phone's.
As each series improves and Apple opens up new libraries so your watch becomes more and more independent of the phone, I believe we'll get more watch developers to up their game.
You don’t actually need the phone nearby with the previous watches either, they will connect directly to WiFi if they need to - they just prefer to use the phone for battery saving.
I was kinda surprised the first time I got an iMessage on my series 2 when my phone was in airplane mode!
Any insight into battery life when collecting data?
I didn't get a chance to profile the app for battery consumption since my recordings were only 5-10s each but I'd assume that while the watch is recording the battery life would be significantly less if you had it on for an extended period of time.
Thanks