AI and special needs adults: The AI Guardian is in sight

Over the past decade I've written on social media about the concept of an AI Guardian, but I realize I've never put it into a blog post [4]. With the release of the LLM AIs [1] including ChatGP4, ChatGPT plugins,  and recently GPT Agents (GPTs) the AI Guardian is much closer than I'd expected. So it's time to write something.

First, of course, I decided to ask ChatGPT. I've never seen the concept of an AI Guardian online, but evidently I've been looking in the wrong places. As of its April 2023 incarnation ChatGPT4 has quite a bit to say:

You: What do you know about the concept of an "AI Guardian" for special needs adults?

ChatGPT

The concept of an "AI Guardian" for special needs adults, particularly those with low IQ or autism, revolves around using artificial intelligence to provide support and assistance in various aspects of daily life. This concept is still emerging, but it has several potential applications: 

Safety Monitoring: AI can be used to monitor the individual’s environment for safety hazards or emergencies, alerting caregivers or authorities when necessary.

Routine and Schedule Management: AI systems can help in managing daily routines, providing reminders for medication, appointments, and other important activities.

Behavioral Analysis and Support: For individuals with autism, AI can analyze behavior patterns to predict and mitigate potential stressors or overstimulation.

Communication Aids: AI-driven tools can assist in communication, especially for non-verbal individuals or those with speech difficulties, by interpreting signs, gestures, or other forms of non-verbal communication.

Learning and Skill Development: AI can offer personalized learning experiences and skill development activities, tailored to the individual’s needs and learning pace.

Social Interaction Facilitation: AI can help in guiding and facilitating social interactions, which can be challenging for some individuals with special needs.

Health and Wellness Monitoring: Continuous monitoring of physical health, including vital signs, sleep patterns, and physical activity, ensuring prompt response to health issues.

This concept is still in its nascent stages and raises important considerations regarding privacy, ethical use, and the need for human oversight. It's crucial to balance the benefits of AI assistance with the dignity and autonomy of the individuals being assisted. As a physician with experience in health informatics [2], you'd appreciate that while AI can provide valuable tools, it cannot fully replace the nuanced understanding and empathy provided by human caregivers.

Well, that covered the bases. I would love to know what ChatGPT4 has been reading on this topic.

In our family iPhones and narrow focus AIs like Google Maps have been both an essential tool for independence and a screen addiction curse [3]. The ability to locate our children was key to them learning to navigate by bike and bus. Mapping tools compensated for very limited navigation ability. Calendaring helped with anxiety and organization. Notes and Reminders helped with memory and recall limitations. Beyond the obvious advantages of messaging we discovered both our children were better able to process text messages than verbal communication -- especially for difficult topics.

Now there is the near term prospect of far more support going forward. Monitoring email, messaging and apps for the scams and frauds that live on the cognitively disadvantaged. Monitoring financial transactions for worrisome trends and events. Monitoring home environments for worsening or dangerous conditions. Encouraging better diet and activity.  Acting as a job coach. Appropriately paced and reinforced lifelong learning. Guiding our children through the vast complexity of medical care. Even providing a form a companionship in time or facilitating finding company in the physical world. Doing, in other words, what Emily and I do after we are incapacitated or (inevitably) dead.

In the absence of personal immortality for Emily and I the AI Guardian is the best hope for our children's longterm health and happiness. Even as I have profound concerns about the disruption that comes with the rise of the modern AI I also have hope for a better options for the cognitively disadvantaged.

- fn -

[1] Some dislike applying the term "AI" to Large Language Models because they don't appear to be conscious or sentient.  Much of my professional career was loosely related to precursors of modern AI and there is absolutely no doubt that in 1995 we'd have called ChatGPT a bloody scary AI. The goalposts are mobile. LLMs are AIs by any reasonable definition. Insofar as we understand human cognition we do much of the same things that LLMs do.

[2] This "as a physician" bit startled me until I remembered that I'd given ChatGPT4 that information as a persistent context for our sessions.

[3] I tried writing a book on how best to balance these harms but I gave up when I finally accepted that there was absolutely no interest in such a book from the community I was writing for.

[4] A 2015 post on Google Inbox was the closest I could find on the topic of AI assistance. I tagged that nsAI for non-sentient AI.

Smartphone for All: Facebook risks - predators and porn

The Facebook chapter of Smartphone for All keeps growing. I think I’m over the worst of it. I’m hopeful it will be particularly helpful, it’s going to feature in a May presentation I’m doing in the Twin Cities.

 One of the tricky sections was figuring out what to say about Facebook pornography. Here’s my current draft, I may move the “why Facebook anyway” part to a different section …

Social media is risky for everyone. Many professionals either abstain from Facebook, use a pseudonym, or read but never interact. So why am I writing a chapter about Facebook use?

I’m writing about Facebook because many teens and adults want to use Facebook, and it’s much easier to help navigate than, say, Snapchat. Facebook is hard to avoid; it’s the primary way many of us learn about community news, events, and social activities. For many users Facebook is a primary news source.

Facebook is also a social experience. For most neurotypical users it’s only one of many social experiences, but many Explorers have limited social options. For them the social connections is particularly powerful.

Facebook can also be an opportunity to learn about social interactions with a Guide’s help. Especially if an Explorer uses a pseudonym (see below) there’s an opportunity to make social mistakes that a Guide can help with. Many Explorers only learn through experience.

Assuming an Explorer is going to use Facebook, what are the risks to watch for in addition to the social mistakes that every Facebook user experiences?

I know of two related risks that can be a special problem for Explorers and and other vulnerable users: sexual predators and pornography.

I’ve been unable to find any academic or police data on sexual predator activity on Facebook. A 2012 Reuters article^⁠1 tells us have read that Facebook uses AI type software to detect predator activity and that “The National Center for Missing and Exploited Children processed 3,638 reports of online "enticement" of children by adults last year, down from 4,053 in 2010 and 5,759 in 2009.” Although only a fraction of incidents are likely reported the downward trend is encouraging. Facebook is probably relatively risky territory for predators, though even one can do a lot of damage^⁠2. Every Guide will need to measure this risk for both male and female Explorers, but as social networks go Facebook is likely safer and easier to monitor than most.

Pornography is a more complex problem. Facebook’s March 2016 terms of service^⁠3 say “You will not post content that: is hate speech, threatening, or pornographic; incites violence; or contains nudity or graphic or gratuitous violence.” In practice Facebook relies entirely on investigating complaints, it doesn’t actively seek exceptions. I’ve seen Facebook investigators decline to act on (closed) Group content that flagrantly violated the terms of service.

Whatever Facebook may claim, as of 2016 anyone actively seeking pornography on Facebook will find it, either by information exchanged at school or through Facebook itself. Of course most teens and adults won’t bother to look, if they have unrestricted web access they will find a vast array of pornography elsewhere. Facebook pornography is really only a problem for users with Facebook access but restricted web access, including children and many Explorers.

Some Guides will, because of personal values or Explorer issues, want to monitor and block extended access to Facebook pornography regardless of related problems (there’s no way to prevent initial access). Other Guides and Explorers may not be concerned by pornography alone. Unfortunately there are two related problems that favor monitoring and restriction.

One problem is that Facebook shares a lot of data among Friends, including an Explorer’s Friends, Groups^⁠4, and Likes — not to mention their posts and comments. An Explorer may unwittingly share Facebook pornography with grandparents, siblings and friends.

Another problem is that nobody creates pornography as a charitable enterprise. Facebook pornography has to make money, and since it’s technically banned it can’t rely on the usual advertising or game revenue. Facebook pornography has only a few ways to make money, including inducing Explorers to install ransomware^⁠5 and other forms of malware.

Until an Explorer advances to unrestricted web access, it’s probably a good idea to monitor for pornography delivered through Facebook Groups, Friends, and Pages and to work with an Explorer to remove the offending items. A Guide may choose to report issues to Facebook, but the results can be disappointing.

 

¹ http://www.reuters.com/article/us-usa-internet-predators-idUSBRE86B05G20120712

² http://www.telegraph.co.uk/news/uknews/crime/9275731/Facebook-sex-predator-jailed-and-banned-from-using-the-site.html

³ https://www.facebook.com/terms

⁴ In theory only Public Groups. In practice information on closed groups can leak out as well.

⁵ Ransomware encrypts a user’s storage device and demands cash to make user data available. As of 2016 it’s a very profitable business.

Managed identity for the special needs Explorer: iCloud makes aliasing easy

From Smartphone for all …

When you first begin supporting your Explorer a single address for them to send and receive email is fine. You may find, however, that you want another, secret, address for your Explorer. You might not want to expose their private email address when signing up for Facebook for example. You may not want them to be able to reset their bank account password. With experience you’ll find many occasions where want to use an email address for an Explorer that sends email to a Guide.

There are three ways to create this kind of email address for an Explorer, depending on what the Guide is familiar with.

If the Guide is using iCloud things are very simple. iCloud email has elegant support for up to 3 “aliases”. A guide can create an “alias” for an Explorer; that email becomes the Explorer’s “managed identity”. Here’s an example of an alias created for a famous Explorer - Meriwether Lewis of Lewis and Clark fame:

If Meriwether were my Explorer and I configured this alias on my iCloud email then I would receive email sent to mlewis_ident@icloud.com (careful, someone is sure to create this email address, I don’t use it!). I might do this so I can monitor Facebook activity and control password resets for example. (see Facebook Social and Facebook Messenger). See Apple’s support note for more information: https://support.apple.com/kb/ph2622.

If a Guide is using regular Gmail they can’t create an alias. They can, however, open a new Gmail account for the Explorer and set it to forward email. This is awkward and requires yet another password to manage, but it only has to be done once.

Lastly, technically oriented Guides will know of other email redirect options that are beyond the scope of this book. (I don’t recommend using a free redirect service however, there are too many security risks.)

All of these approaches will work to create this extra, secondary, email address. This secondary address doesn’t get entered into your Explorer’s phone, it’s just for use by a Guide, typically though a web browser like Chrome. 

You don’t need to add this second, Guide-use only Explorer email at first, but keep it in mind as you gain more Guide experience.

Smartphone for All: Examples of using Apple Notes.app or Google Keep.app to extend memory and independence

A Smartphone for All: book excerpt, from a chapter on using Notes. 

This post has been updated with some excellent additions by Deb T. In addition to being a part of the book, it also illustrates how complex a special needs adults independent living really is.

————

Whether an Explorer is following the Apple Way or the Google Way their notes will look very similar. They’ll typically start with a handful of Notes created by a Guide, but the number will grow over time. Some Explorers will add their own Notes. 

Most Explorers will prefer to browse Notes rather than search for them. Typically Notes will be ordered by the date they were last changed with the most recently changed Notes at the top. The first line of a Note will typically show as the Note title, so make it descriptive.

If an Explorer is using the Search ability they are probably also creating and editing their own Notes. When using this function it’s helpful to put likely search terms in the note, perhaps as a list of words at the button of a Note.

Notes can be organized into named collections called “Folders” in Apple Notes.app or organized by “Labels” in Google Keep.app. This adds complexity however and should only be considered if an Explorer has more than 50 notes and really dislikes using Search.

There are many ways to use Notes to extend an Explorer’s memory. The following table gives a few examples taken from real world experience; this table also shows how complicated an Explorer’s routine can be.

Many of these Notes hold non-sensitive or public information, but some require that both the Explorer’s smartphone and Cloud information are truly secure. We reviewed this in Setting up an Explorer’s Smartphone including an encrypted smartphone, long letter-number smartphone unlock codes, fingerprint identification, a responsible Explorer, short timeout auto-lock, and a strong Cloud password.  Some low end Android phones may not be encrypted, don’t put confidential information into Google Keep on those devices. In the table below items that require a secure device are italicized.

 

Note Title	What’s in the Note
Banking information	Bank account information, how to make deposits or withdraw money or find balance, ATM PIN code if Explorer has difficulty remembering it, numbers to call for a lost or stolen credit card.
Bicycle	Bike maintenance advice, serial numbers, what to do if lost or stolen.
Budget	Notes about when bills are due, how to pay, wise limited on spending.
Church, Temple, etc	Names of religious leadership, times of services, people who are part of religious life, social events coming up.
Clothing sizes	Sizes for clothes, boots, shoes.
Combination locks, PINs and passcodes	We all have too many of these. A single Note is a good way to hold them all, but of course this requires a secure smartphone and a secure Cloud.
Dates	Important dates, such as birthdays and anniversaries. (Duplicates what’s on the Calendar, but often useful to have separately.)
Emergency Information	Who to contact in case of an emergency. This is not the Emergency information that’s part of Apple Health.
Family photos	How to view family photo shares.
Financial worker	For many Explorers the Financial Worker (benefits admistration) is separate from the Social Worker and there are different procedures to follow. Some Explorers will want Notes combined, others will like separate notes.
Fun stuff	Notes to support local recreation, leisure and fun activities; a helpful resource for independent time scheduling. An Explorer or Guide may use this Note when working on the Calendar. For example: movie theater, pizza and sub delivery, church, etc. Gym information might go here instead of the Gym Note.
Gym information	Open hours, class schedule
Home	Apartment/group home details including contact numbers (also in Contacts), look out procedure, how to request maintenance, how to work with the rental office.
Library information	Schedule, library card number, WiFi procedure.
Medical history, dental and health insurance numbers	To share with caregivers, particularly in an emergency. Includes major medical problems, current medication, providers, people to contact, insurance numbers.
Neighbors	Names and addresses of neighbors, particularly for Explorers who have difficulty remembering names or matching names to people. This may duplicate what’s in Contacts.app (address book) but is helpful to have separately.
Passwords and credentials	Username, passwords, web site address (URL), “Secret questions” with the answers used.
Relatives and special people	Names and birthdays of extended family and special family friends. May be combined with Neighbors.
Residential program manager/staff contacts	Some of these will also be in Contacts, but this is important information for many Explorers. It deserves a separate Note.
Smartphone tips and how to	Basic smartphone tips. If appropriate for Explorer a reminder of how to find the smartphone manual (example: iBooks).
Social worker	Name of social worker and how to work with them, particularly around bus and transit services. The Transit Note might refer here, and the Social worker information might also be in Contacts.
Social Security and Disability Information	Contacts and details, spending limits and reporting requirements.  Date of next follow-up (this would also go in the Calendar).
Sports teams and social group	For each sports teams and social group useful information such as link to team calendar, names and numbers of coaches and players.
Staff and aides	Names of support staff and contact information. This will also be in Contacts.app
Transit	Guides to transit including bus pass information if applicable. Reminder of how to use a transit app to get bus information.
Travel and packing	Packing and travel advice for a short trip.
Web addresses	Generally web addresses will be part of other Notes or they’ll show up on the Credentials page, but in some cases it’s useful to have a short separate list.
WiFi locations and WiFi passwords	Explorers need WiFi to backup their smartphones or download new apps, but they may not have WiFi at home. In some cases it may be useful to list locations and passwords, such as Coffee shops or the Apple Store. Library WiFi information may go here or in the Library Note.
Work and work training program.	Supervisor and care manager and any work requirements or procedures including transportation arrangements.

 

This is a long list, but the the more independent an Explorer is the more their Notes collection will grow. A Guide should start with 4-5 Notes then work with an Explorer to build out the Notes collection.  Many Notes may begin as information sent in an Email; very few Explorers will search email for reference information.

Some Note content overlaps with Calendar and Contacts. A moderate amount of duplication is needed, but too much becomes a maintenance burden. The appropriate location will vary by Explorer, but the Calendar is particularly important.

Most Explorers will learn to update and manage their Notes; that is a great life skill to encourage. Managing Notes includes deleting obsolete Notes, a Guide will want to make Notes review, including deleting obsolete Notes, a part of their scheduled weekly Explorer support time.

Autism and interest depletion - leveraging routines, calendars and checklists

As #2 enters late High School his interests have narrowed considerably. This means he has fewer options if he bores of the interests he does have, and increasing amounts of his time are spent in passive and compulsive screen activities that seem to produce dysphoria and ennui rather than satisfaction or happiness. It’s a common trend that seems unlikely to lead to a satisfying or independent life.

Fortunately, he’s aware of this and, when he’s not working through what I think of as an “autistic-arrest” (sudden deterioration in perception of self and context, often associated with anguish and psychic distress), he’s interested in working on it. He’s had some success, including introducing new activities like mountain biking, weekly five mile walks and routine calendar review. We’ve created an inventory of his interests categorized by:

1. Enjoys with cravings -- time limited by parents/external actors
2. Enjoys with cravings -- no time limited (bouncing on exercise ball in in this group)
3. Enjoys but no compulsions — tend to get displaced by craving activities, but if initiated they are enjoyable for him
4. Sometimes enjoys, feels good after doing. All exercise except bouncing falls into this category.
5. Things to try — items we (he, us) have selected to try to broaden his interests and provide him with more options.

As per the book excerpt on calendaring, we’re working with him to create recurrent schedules for his Category 3-5 items, staring with book reading. He’s receiving a small stipend for each book completed; he finds it very helpful to have us give him a concrete reward for completion of a new or challenging activity. It’s a curious part of his temperament that we’ve all learned to leverage.

Our thinking about this has been supported by some archived 2007 writings by Joel Smith, an autism activist. (#2 adamantly refuses to read blogs or writings by people with autism, but I follow Joel’s current blog). Joel was using a PalmOS PDA back then, my book project is about smartphone use.

This Way of Life: Living With Executive Dysfunction (Joel Smith)

… Train yourself to follow a routine, with few deviations. The closer you follow a standard routine, the more likely you are to remember it … The more frequently you use a routine, the more likely it is to become a habit…

… solving the executive function difficulties probably won't solve the problem without you first solving the other underlying problems. In addition, your strategies will not motivate you to do a task you don't want to do. These strategies will only help you perform tasks that you want to do…

… Get rid of strategies that don't work. If, after a week you aren't able to establish the strategy, it probably won't ever actually work. If, after a month, you are still doing the strategy, you will probably be able to keep doing it…

… Sggab (amanda at autistics.org) finds that having someone else to watch over her helps her to complete tasks and overcome the problems with starting a task. Some people find that having others call and remind helps get a task done. However, others claim that such reminders only reinforce inertia, so this may be very individual specific…

… Kalen (news at paradox.freeserve.co.uk) says that an externally imposed schedule, such as the kind of schedule a student in formal education must follow, helps her start all her other routines, and also improves her overall functioning…

… I use a Palm Pilot to help organize my time. Even the cheapest Palms, such as the will help some who have executive function difficulties.

The most useful features of the Palm are the to-do lists and calendar. One of the advantages of some of the newer models is the "silent alarm" feature. Reminders can be set to alarm by vibrating instead of beeping, which allows you to get your reminders discreetly and without disturbing others in a class or event.

I also use an enhanced version of the Palm ToDo list, called “ToDo+”...

The entire article, and his reminders and checklist page, are well worth reading for other approaches to routines like eating, dressing, taking medications, paying bills and the like. I’ll be referencing his advice in the smartphone book.

Book chapter excerpt: Smartphone calendaring and special needs

For the book. Comments welcome …

Calendar management is one of the most powerful smartphone tools for independent living. Knowing what to do when takes a lot of memory stress away. The Calendar is where healthy and helpful habits are maintained. It can be very hard for spectrum Explorers to initiate new work, scheduling it on a calendar can be a good launch step. Special Hockey games, doctor’s appointments, work schedules, study schedules, screen break times, family outings, dates, holidays — it all starts with the Calendar.

That was true in the days of the family wall calendar, but the smartphone Calendar is a big improvement. It’s always at hand, appointments have reminders, and, best of all, it’s easy for one or more Guides to manage an Explorer’s Calendar.

Calendars are powerful but relatively easy to use. Every Explorer able to read at a 3^rd grade level will be able to use a smartphone Calendar app. The key is to practice and reward Calendar use. The “Guide’s Toolbox” chapter described a general approach, with the Calendar there are some simple specific techniques. One is to respond to most “when” questions by asking your Explorer to access and review their smartphone Calendar.  Another is to reward routine daily review of the current and next day’s Calendar. Some Explorers may be stressed by the imposed structure of formal Calendar, in this case you will need the usual mixture of patience, reward, pressure, support, persistence, strategic retreat, reframing, empowering and more persistence. Just like everything else that’s new and important.

Of course this assumes that you are also using a Calendar effectively! Actions and examples always mean more than words alone. You’ll want to be comfortable with your own Calendar use before you introduce Calendaring to your Explorer.

….

Apple and Google take similar approaches to Calendaring, but there are some important unique advantages for each one. 

Both Google and Apple have Calendar apps that work well with your smartphone. Both allow a Guide to remotely manage and review the Calendar from a web browser, including adding new items remotely. Both Google and Apple support invitations to Events, Event locations with integrated maps, and calendar overlays. Calendar overlays are pretty useful, they let users see several people’s color-coded events in a single integrated calendar. 

On the smartphone both Google and Apple are building “intelligent agents”, like Siri and Google Now, that use Calendar information to create reminders and suggestions.

Apple’s Calendar solutions have an advantage of simplicity — as long as you stick strictly to “iCloud” and don’t try to subscribe to any “Calendar feeds”. In my experience few people use Calendar Feeds, so as much as I personally love them this may not be a great problem. Apple’s Calendar solutions also work well even when an iPhone doesn’t have data services.

Google’s Calendar solutions are more powerful and far better at working with Calendar Feeds, not least because many schools, businesses and organizations use Google Calendars…

See also: Autism and interest depletion - leveraging routines, calendars and checklists.

My book to be: The special needs smartphone for independent living

I’m working on a book with a working title of Special Needs Smartphone. I don’t think that will be my final title, but it’s a good description. It’s a book I’m well placed to write; I have years of experience with two very different special needs smartphone users, I’m a geek, I’m used to writing, and I have the opportunity to do it. I’m not expecting to make money, but I do hope to make a positive difference. The book will cover both iOS and Android (that’s not easy, by the way :-).

I’m writing this book for parents and other supporters of teens and adults with a range of different minds. This includes teens and adults with IQs of 60-70 and limited reading skills, but it also includes normal IQ autism spectrum teens managing disabling preoccupations or aversions to some independent living skills. Some on the spectrum, for example, might find calendaring painful, have an aversion to lists and schedules, or be unfortunately prone to getting lost.

The goal of this work is to support independent living and mitigate harm that can come to a vulnerable population living with a tool of amazing, sometimes frightening, power and versatility.

I’m considering publishing options and a kickstarter campaign, but my default assumption is that I’ll self-publish through Amazon print-on-demand and self-fund. For the community I’m trying to reach I think a printed book is essential, eBook is secondary.

At some point in the next few weeks I’ll launch a domain and blog/tweet under my “true name” to promote the book and support readers. Until then I’ll have some posts here; I like blogging as a way to think about what I’m working on and some of these posts should be useful in themselves. You can get a feel for what’s coming by reviewing the Best You Can Be blog posts I’ve written on this topic over several years (Technology tag). One caveat — the book tone will be lighter and brighter than my usual somewhat dour style (I blame my Scots ancestry).

If you have ideas, advice, want to recommend a publisher or just want email notification of publication, please email me at jgordon@kateva.org or leave a comment below.

I’ll let you know how it goes.

Special needs smartphone: looking for a Notes app with a difference

I’m working on a book about iOS and Android smartphone use by special needs teens and adults; It will have a special focus on supporting independence. The project is in its early stages, but sometime in the next few weeks it should get a companion domain and a blog. In the meantime I’ll have a few posts here on related topics.

One of the things I don’t have an answer for is the “deletion problem”. My #1, for example, hates clutter. On an iPhone that means he compulsively closes open apps, deletes his browser history, old email, saved images, iMessages and even the Notes I’d like him to have. Notes that consist of things that it would be useful for him to remember.

The only thing he doesn’t delete are his Contacts.

From a psychological perspective this is interesting, but since he has significant memory disabilities it’s rather self-defeating. So I’m looking for a solution in the myriad of iOS note-taking / extended memory solutions. My high level requirements are:

1. Plain text support (anything more is either “nice” or unwanted complexity).
2. Search
3. Web client (key to a support person creating or contributing to notes). Native clients are a plus, but the support model I’m using must include Chromebook
4. Efficient backup/restore of all or part of a notes collection from the web client and/or the ability to make some subset of notes read-only for our smartphone user.
5. Simplicity.
6. An exit strategy — ability to move content to another solution if needed.
7. At least a 3-5 year expected lifespan (which is a long time in this market)

Apple’s native Notes.app comes close to meeting my requirements. It even has limited undelete functionality on iOS, but as of Dec 2015 not via iCloud [1]. (I investigated Apple’s legacy support for Google (Gmail) IMAP notes sync, but this does not allow remote authoring or backup and has other limitations.)

I personally use Simplenote; it can be paired with nvAlt on OS X to allow bulk import of a standard set of text files, but that’s a geek-only solution. There’s no backup or import ability in the Mac client.

Anyone have ideas?

- fn -

[1] Apple’s support article on iCloud Notes data backup is a bit humorous. “… Copy the text of each note and paste it into a document on your computer, such as a Pages or TextEdit document. Save the document to your computer.” Must have been painful to write that!

Update: Shawn Blanc’s 2013 post has some good ideas. Reviewing it I see Simplenote eliminated Dropbox sync, and Justnotes.app, which Shawn liked, has been discontinued. Byword, Nebulous Notes, and Notesy use Dropbox as a sync repository; Byword and Nebulous Notes are complex but Notesy might be an option. Microsoft OneNote is powerful and impressive, but also complex.

Update 2: Simplenote.app allows sharing of note by the email addressed associated with a Simplenote account. Shared notes can be collaboratively edited, which is good and bad. On the other hand, if one collaborator deletes a note then they are only removing themselves as a collaborator — every other collaborator keeps the note. So this may actually work reasonably well, with the additional option to use nvAlt (Mac) or ResophNotes (windows) for more robust backup and restores.

Cognitive disability and AI assistance with Google Inbox.app. Suddenly, a new world.

Google announced nsAI (non-sentient artificial intelligence) assisted email today, it will debut in the Android and iOS Inbox.app.

My first thought was that this will be kind of annoying. A few minutes later I was thinking about AI-responses generating AI-responses and the various spam implications. I decided this would be interesting, exciting, maybe a bit scary. There will certainly be unexpected consequences.

Then I remembered how much iOS word and phrase completion has helped #1 son with texting and email. I remembered that I’ve been watching for more nsAI assistance to support both of my sons. 

That’s when I realized how big this announcement really is.

I’ve reinstalled Inbox.app on my iPhone and Ill be planning how to transition both of my sons to the new platform and how to introduce the concepts of nsAI assistance to our local educational system.

This is how the future comes crashing in.

Special needs: so now have to worry about ISIS recruiters?

The New York Times did a good job describing ISIS (ISIL, Daeshite) recruitment of a cognitively limited 23yo woman. She’s not explicitly described as adult special needs, but she has a fatal alcohol syndrome diagnosis and limited employment options. She’s not that different from the kids and adults we care for.

Unsurprisingly her ISIL recruiter is not terribly high functioning himself.

It reminds me of the relatively harmless cults of the 1970s, but in those cults there wasn’t a lot of upside to special needs recruitment The cults wanted fundraisers; a certain amount of psychological disability was a feature, but special needs was too much. ISIL may have more use for people with cognitive disabilities.

It’s not at the top of the worry list, but if you care for a special needs teen or adult with a net connection (smartphone, etc) you can probably add ISIL to the long list of predators of vulnerable adults. Not the top of the list, but something to watch for.

Yay connectivity.

PS. I’ve long wondered what Richard (“shoe bomber”) Reid’s IQ is, and whether he’d fall in the sub 70 range of diminished responsibility.

Special needs adolescence: Yes, Virginia, there is porn on Facebook. How to torch an account.

Young males have been known to view pornography on the Internet. Having a cognitive disability does not necessarily change this behavior. Some kinds of disability, however, mean that discretion can be weak, and boundaries between "acceptable" porn and "unacceptable" porn may not be observed.

That's one reason to try to put some controls in place. Another is that Internet porn is often the equivalent of the sweet smelling sap of a Venus fly trap. It comes with malware, and it can be used to identify a vulnerable population for additional manipulation. A cognitively impaired young person makes a fine catch.

So we have controls -- lots of controls. Not that Apple makes it easy, rumor has Mac Parental Controls are abysmal by design (I so want to kick those engineers). Facebook on the other hand, can be more or less monitored through the User Activity Log. So we've been allowing monitored access, with no password access. In addition to our monitoring, Facebook promises... 

Facebook: "Facebook has a strict policy against the sharing of pornographic content ... "

Of course you'll notice they don't say they actually enforce that policy. Thanks to my diligent research assistant I can tell you that Facebook has a diverse and abundant quantity of porn. It appears that the average Facebook porn site lasts a few weeks before it's shut down, but there are always thousands of new sites to fill the gap. I assume the creators are using the Venus Fly Trap business model.

So much for Facebook. The trick then is how to manage the Facebook account - now known to the scum of the earth. In this case I used the User Activity interface to "unlike" the relevant pages (might be useless, but seemed a good cleanup step). Then I used Facebook's obscure Account Deletion page (https://www.facebook.com/help/delete_account‎) to torch the account.

Sometimes experiments fail. So we take a break and try something different...

My son learns algebra on his iPhone via DragonBox

Friggin awesome.

That's what I got from the look on #1's face when he solved an equation with x on one side, and a numeric expression on the other. He was doing algebra, he knew it, he was proud.

Damn, that was the best $6 I've ever spent.

#1 is entering 10th grade next year in the special needs modification program. He reads at about a 4th grade level (perhaps less) and struggles with basic arithmetic and time calculations. Despite years of practice he can't do long division by hand. Despite this reality, he declined the standard transition programs for a regular junior high academic schedule -- though we worked him down from three languages to one.

Algebra is one of the items on his junior high dream list. That seemed reasonable to me. We weren't getting anywhere with our old arithmetic drills - he was bored and frustrated. Further progress will require years of slow practice on real world problems, and use of his iPhone calculator. Seemed a good time to try something different. So, for his summer homework, we decided to try DragonBox, a French-Norwegian iOS/Android/Windows/OS  X math "game" we'd used with my daughter on an appnetizen's advice (Thanks Jonathon!).

He's been doing it for summer homework several days a week. Sometimes he'll randomly flail at it until the problem is solved, but watching him I could see something else developing. Something that perhaps played to his near-savant visual processing strengths. With DragonBox he was learning negation, reduction, balanced operations, add zero, divide one - basic algebra. A little bit of progress every day - on his iPhone, as a game.

Today he got to the level where the graphical icons were replaced by numbers and variables. When he solved the numeric expression he blushed with pride and joy.

That's worth $6 I'd say.

Now we'll work with transferring these skills to paper. I will do paper operations in parallel with his app operations, then see if he'll replicate on paper what he does in DragonBox (step-by-step), and so on. Maybe it will work, maybe it won't, but in our family we believe in declaring victory early and often (and so life is an unending series of wins, until we die).

I've bought the high school version of the app for #2 and #3 - and I'm following the We Want to Know team's blog.

See also:

• Adaptation versus Modification: Critical code words in understanding K12 special education. (plus Settings)
• Status June 2013
• The end of High School, the end of dreams: transition hardness
• Apple's iPad/iPhone App store has a special education section: It's still there
• Math fact drills for an Asperger's child - two excellent solutions: worked well to a point.
• Be the Best You can Be: Judo moves on an atypical mind: Plan iMac: 1/2010. This one worked ok, but he mostly uses the Learning account when he can't have phone time.
• Different minds, Different paths: 10/2009. I'd forgotten how well he did with the game "rush hour".
• Adolescent special needs: Sometimes Judo works 3/2013

Net world and guiding the special needs adolescent - tricky business

#1 is less than two years away from legal adulthood. In his case that will mean guardianship, but more independence nonetheless.

So he gets more freedom, including an iPhone Facebook.app account I can monitor. He doesn't know the account password, I entered that into Facebook.app on his 2nd hand contract-free $10/month Paygo iPhone.

We review his Friend list and activity about 1-2 times a week, and of course I visit his page from my Facebook account and from his account. It's not foolproof of course -- like most 3rd party iPhone apps Facebook.app ignores the iOS Safari setting and allows unfettered web access via embedded Webkit. (Yes, Apple's parental controls are a joke. Android's are even worse.) At his age though, even that backdoor is a bit of a feature. It lets me more or less observe how he handles it.

So far, so good - but there's a twist. He likes to save images he finds on the pages of his High School friends and acquaintances. They are views into a world he cannot join; I grieve for that. I think, in his own way, he grieves as well. The pictures are generally benign, but they are pictures of 15-18 yo boys and girls. He is prone to share them via his iOS Photo Stream to his brother and I.

And that's the twist. Pictures that are reasonably appropriate for the phone of a 16yo boy are not a great idea for the phone of a 50+ yo man. It took a week or two for me to think of this and turn off his photo stream. I'll have to simply review his phone, which is easy to do each night.

Tricky business being a special needs parent in the net age.

Adolescent computing and OS X Parental Controls - training wheels

My 14 yo's computer skills have continued to be a real strength. Of course, being both 14 and having disabilities in executive function, he does not always use them wisely.

Years ago I hoped the iPhone and other iOS devices would provide app-restricted services while limiting web access. Sadly, I've been disappointed by Apple's deceptive iOS "parental controls" [1]. That didn't work very well. On the other hand, monitoring his computer use and punishing misuse isn't working that well either. We can't be looking over his shoulder everywhere -- such as in his school room.

So now I'm trying Plan B, an educational program of trial and reward based on techniques that have worked before.

I've set up an account on a machine using OS X Parental Controls [2]. I've whitelisted a number of sites he's interested in, including bing.com [3]. I give him 10 minutes every day to use the machine without me watching. When he's done we review his history and log files together (in Parental Control mode history files can't be wiped -- he learned that trick at school).

If he's used the computer responsibly, he gets some reward tokens. If he hasn't he doesn't get any and I make the whitelist more restrictive. Since I'm trying to extinguish the bad behavior I keep things dull.

He's keen on this, and so far he's doing well. My plan is to keep broadening the list and reducing the parental controls, rewarding self-control over time.

I'm sure it won't fully succeed, but at the very least I'm hoping he'll learn some discretion. If he can think for a second before acting, he may avoid a lot of trouble.

[1] I'm trying various channels to get Apple to improve, including contacting legislators and this email to Consumer Reports ...

I'm writing to ask you to include 'parental control' functions in yuor next evaluation of smartphones.

We've been very disappointed in the parental controls Apple provides; in some ways they're worse than nothing. It seems we can limit web access by disabling Safari, but many, many apps contain embedded browsers. Once a child has access to an embedded browser, they can usually find their way to a Google search and abundant porn.

Embedded browsers are found in encyclopedias, MLB (major league baseball), reference works, educational materials, and many others.

The fix is simple. Apple needs to provide a parental control that restricts embedded browsers. Until they do, iPhone parental controls are a deceptive fake.

If you include parental control functions in your evaluations, Apple and other vendors will pay attention.

[2] They're mediocre at best, but they're (barely) enough for this experiment. See updated documentation below.
[3] Though Bing's parental controls are even worse than Google's he's less familiar with Bing so he doesn't dodge them as readily.

Update 5/29/11: This is working better than I'd hoped. The huge advantage of Bing.com over Google.com is that Bing doesn't use https. So OS X Parental Controls show a meaningful history of sites visited. I'm going to explicitly block Google.com but permit Bing.com.

Update 6/12/11: History files cannot be wiped in the most strict form of parental controls. In the "automatic detection" mode they can be wiped. See Gordon's Tech: OS X Parental Controls Review - State of the art in OS X 10.5 and 10.6.

Apple's iPad/iPhone App store has a special education section

The iPhone/iPad App Store has a section devoted to special education. I was able to find some announcements from the end of October, so it's quite new.

There's a lot there, from sign language to communication to accessible readers to language development apps.

This opportunity to market and sell focused special needs apps could be a very big development. I'm excited, I've written before about our own experiences with my son's iPhone, including the weaknesses in iPhone parental controls. He's probably moved beyond most of these offerings, but we'll be examining them in more depth. (Lately he's been using the money to earn to buy games, which is an improvement on using it to buy candy.)

Understanding a different mind: memory organization and receptive language

As my oldest son moves into his adolescence, his mind continues to change. Observing him, I get new insights into how his mind works.

He has a pen pal now, a young woman who is studying special education. She started writing him as part of a school program, and has continued on. She is a wonderful correspondent.

My young adolescent tells her stories to impress her. They aren't, however, true stories.

They are generally plausible stories, no more or less impressive than the things he actually does. Often they are things he has done, just not things he has done recently. On the other hand, he omits adventures that I, in his place, would certainly include.

I think he's dissembling a bit, but mostly I think he doesn't really remember what happened yesterday. He may remember it in detail six months from now, but at the moment it's lost to him. So, like an Alzheimer's patient, he fills in the gaps. He tells a story.

His memory dysfunction is a profound handicap all by itself.

His inability to process speech is probably related to this memory dysfunction. It's not usually obvious to anyone, even to us, but he struggles to process  even relatively simple sentences. He will often react negatively to a request, but, if he's given a minute or two to think about it, he will usually reverse himself. I think he needs time to try to reconstruct the sentence, to understand its meaning.

Curiously, as his reading slowly improves, largely due to his email and texting, his written communication is becoming stronger than his verbal communication. He understands ideas that are written as words better than the same ideas spoken aloud. He can read and reread printed text, gradually building understanding and reinforcing his limited short term memory. Spoken words are hard to reprocess.

He has other disabilities, but also some useful strengths. He's quite good with his iPhone, and with software and devices of every variety. Already his calendar, which I can edit from my desktop, is turning into a history of what he has done as well as what he is going to do. He takes hundreds of pictures of construction vehicles (an obsession), but in time the phone photo library will become a history of things he has done, places he has been. We often discuss things by text message; I think he finds that easier than spoken communication.

The phone record, including a record of where he and his phone are, may have other value as he gets older. Many cases of false imprisonment seem to involve people like him, people with such disordered memories that they can choose guilt for any crime. His prosthetic phone memory may protect him.

Google Scribe for special needs writing?

I'm typing this post with helpful from Google Scribe. As I type I get a popup list that lets me type a number or hit enter to autocomplete my sentences. It even suggests phrases based on what I type. I used to use something like this on my old Palm III; it was helpful there but it's somewhat distracting when I have a keyboard.

Lee Matthews has a good description ....

Google Scribe bookmarklet puts powerful autocomplete anywhere you enter text on the Web

... Ever wish you could leverage the power of Google Suggest anywhere?...

Those users might want to check out Google Scribe, a slick new Labs offering that brings Google's suggestion magic to any text entry field on the Web. Just visit the Scribe page and drag the bookmarklet onto your bookmarks bar, head on over to your favorite social site like Facebook or Twitter, and give it a go!

Once you click your bookmarklet, you'll see an activation notice appear. As you type, Scribe will display a drop-down list of suggestions, just like Google's search box does. You can press Enter to apply the first suggestion or 1-0 to pick any option in the list (or use your mouse if you prefer). Google Scribe also offers sorting options, and you can set it to "always on" or "on demand" mode...

For my test I installed the Google Chrome scribe extension, but I believe the bookmarklet will work with Safari. You hit Enter to accept the top suggestion with a space, a number to choose another, Esc to hide suggestions and tab to choose without a space.

I think it will work even if you're not logged into Google, though I expect a future version will learn from the words someone uses.

This technology is obviously helpful when doing small device text entry, but for a special needs writer with limited spelling skills it may be very interesting at all times. I'm going to try it with my son the next time he's working (under intense direct supervision!) on his Facebook page.

iPad, iPod Touch and iPhone in autism

When the iPad was first unveiled last January, one excited blogger wrote ...

Gordon's Notes: Computing for the rest of us: The iPad and the ChromeBook

... Think about your family. If it's big enough, your extended family will have at least one person who's, you know, poor. They may have cognitive or psychiatric disabilities. Or you may have a family member who, like most of American, can't keep a modern OS running without an on call geek. These people are cut off...

By 2011 the combination of a $400 iPad (and iTouch for less) and $15/month VOIP access will start to replace a number of devices that are costly to own and acquire, while providing basic net services at a rate that other family members can subsidize. Not to mention something pretty, which, speaking as someone who grew up poor, ain't a bad thing.

Steve Jobs - friend of the poor and the outcast. I wouldn't have guessed (ok, so I did predict this a year ago)..

Since then I've noted how well an iPad can work in practice for special needs persons (most elderly people are special needs, but don't tell them that) and I've configured an iPad-mini (aka an iPhone) for an adolescent with autism, ADHD, anxiety and low IQ.

Now we're reading slightly breathless reports of magical interactions of autistic persons with the iPad environment. I suspect there's some hype and wishful thinking going on, but I also believe there's some reality. These platforms are bringing communication and support technologies to populations that were previously unserviced. They are also making it possible for neurotypical non-geeks to deploy and maintain these tools at a price most families can afford.

The current iPad generation still require a desktop computer (relatively modern PC or a Mac). That's a serious roadblock to widespread deployment. There's no technical reason for this, but there may be business reasons. Apple may be unwilling to wipe out their legacy devices sales just yet. I think there's a 70% chance, however, that iPad 2 will not require a desktop device.

If you have the money and the geekiness, there's no real issue with buying and using an iPad 1.0 with your special needs person of any age (do read my writeup on the iPhone for special needs adolescents though). For most people, however, the iPad 2 or iPad 3 will be more interesting. You want to watch for a device that doesn't require a legacy desktop machine. (In our new world, Windows 7, for example, is legacy.)

 

Adolescent computing - the iPhone and iOS solution

I've written over 30 posts about computer use and the low IQ special needs child. Now I'm dealing with computer use and the low IQ ADHD/Explosive special needs adolescent. This has some special concerns.

One is vulnerability. Most special needs adolescents are vulnerable netizens, susceptible to abuse, fraud, and manipulation. This is also true, incidentally, of many neurotypical adults and most elderly adults.

Another issue is judgment and self-control. This is an issue for any adolescent (was for me, anyway), but for the "explosive child" external restrictions are particularly important. Unfortunately recent changes in net technology (https, multiple data streams, etc) have broken a lot of parental control software. OS X parental controls, in particular, are utterly broken.

Even as one computing platform wanes, however, another waxes. The iOS (iPhone, iPad, iPad Touch) platform is a better match to the needs of many special needs children and adolescents. You can read about one iPhone setup through this post and related links.

The first steps are to active iOS "restrictions", including disabling Safari, YouTube and App installation. The next steps are to add apps that further the parental agenda, while engaging your special needs child. These are some of the apps we use (note despite his disabilities my son has a knack for software, not all persons of similar IQ will use the same apps):

1. Calendaring: His iPhone Calendar app shows our family calendar, school calendar, and his calendar as well as birthdays and holidays. He is very anxious when he doesn't know his schedule. The calendar anchors him. He can edit it, he can see what's coming up. All calendars synchronize to Google Calendar feeds. (See setup link, above).
2. Contacts: I edit them for him on our Google App domain. They help him remember people. I will show him how to add pictures to them.
3. Notes: He likes to make notes in the basic app. I will introduce Dragon Dictate soon to make note taking even easier.
4. iPod: His music of course, but also all loaded with ultra-high quality educational documentaries such as BBC's 2001 documentary "The Blue Planet" ($6.40 for 8 hours). When he's dying of boredom he can now watch TV even without a data connection. Some TV, that is. (Cue evil parental laughter.)
5. Camera: He loves to take pictures. I do need to monitor this. Not all his pictures are equally appropriate.
6. Maps: Learning his environment.
7. Google Earth: obviously
8. Weather, Clock (timer is valuable), Light, Calculator, Voice Memos -- all frequently used.
9. NYTimes - he reads the sports (follows the Mets).
10. Public Radio - so far he hasn't used it.
11. SkyVoyager and Star Walk: wishful thinking. He won't use these without some thought from me.
12. Games -- all serving some educational end - Checkers, tChess Lite, UNO, Solitaire, Virtuoso (piano), Matches (memory), MemoryMtrx (short term memory), Mental Maths and Math Drills.
13. Wolfram Alpha and Wikipanion: Homework aid without Google search.
14. MobileMe iPhone location: we know where his phone is, anyway.

In future posts I'll update and revise the list of helpful apps.

Update 8/10: I received a comment praising the use of a browser product, "mobicip" that claims to substitute for Safari on the iPhone.

Unfortunately there are two reasons to be concerned about this product.

The first is that comment was fraudulently submitted. It purported to be from a user, but "DrJim" was a link to the vendor site. The comment appeared to be machine generated. These techniques violate Google's terms of service, and they are a good marker for a fraudulent enterprise.

The second reason is that mobicip's pricing is deceptive. In addition to the product purchase you must sign up for an annual subscription to get the "whitelist" functionality you truly need.

A NYT times bits post has the best available information on this topic from a source I trust. In addition to mobicip they mention:

• iWonder Surf for $15: includes whitelist abilities
• Safe Eyes Mobile for $20: they also make desktop apps

If I had to choose one I'd look first at iWonder Surf.

Update: In the space of a week this $@! kid of mine has found two loopholes in the security I set up. He discovered that Google Earth has a wikipedia layer, and that clicking on those links triggers an embedded browser than runs when Safari is disabled. He eventually found ways to navigate to places he's not allowed to go. Then he found that a suite of utilities included, for no good reason, the ability to run iGoogle within the app.

Sometimes I wonder about quite low IQ test results.

Thus far he's been so delighted in his discoveries that he shares them with me. I reward him with for the invention, but then delete the app. (Good-bye Google Earth.)

Update 8/28/10: Wikipanion bit the dust. He found some interesting topic pages and pictures. Too bad there's no World Book app. A surprisingly large number of apps have embedded WebKit browsers that are not disabled when Safari is disabled. Apple needs to provide parental controls for disabling WebKit use. I wish Apple would hire me to run their parental controls effort.

Update 9/20/10: #$#$%! Public Radio.app has a $#$@ embedded web browser. Forget the thesis of this post, Apple's iPhone parental controls are as broken as can be. The good news is my son delights in showing them to me. When you have a sub 5th percentile measured IQ, and read 3 grade levels behind, it's a real confidence builder to show your Dad you've outfoxed his security.

Update 10/31/10: Today he was caught again. This time he'd learned to use the New York Times.app search function, then to click on a hyperlink, then to escape to full web search. I've stripped every non-Apple app from the phone. This really is a serious problem.

Update 9/20/2013: Many years later, iOS 7 finally fixes the webkit hole.

Cognitive evaluation and motivation - trickier than it looks

One of my sons has substantial measured cognitive disabilities including base IQ and a range of social functions. By most recent evaluations he's borderline "mentally retarded". (A nasty phrase that's enshrined by legal statutes. Of course there's no true binary state, this is all continua.)

Which is why our titanic struggles over his misuse of internet resources are puzzling. This ought to be the mismatch of the decade. In every measure of knowledge and cognitive measurement there should be no contest between him and me.

And yet it is a struggle. Mostly I win, but he wins some too. He's proven OS X Parental Controls, for example, are utterly broken. (I have more to write about iPhone for special needs adolescence. There's more promise there, starting with disabling Safari and YouTube.)

Yes, he has a knack for software. It's not savant level, but he might be 60% for age -- that's far beyond his other cognitive skills. It's not only that however. He can be very inventive in solving the problems I create for him.

It has something, I think, to do with his alien motivations. He's not particularly motivated by neurotypical concerns like parental approval or even peer approval. This means he barely exerts himself in measurement settings where most people would at least make a go of it. Where he is motivated, therefore, he performs significantly better than his testing would suggest possible.

Of course in most domains he can't perform in the neurotypical range no matter his motivation, but the reality of his peculiar (and highly problematic) motivational structure does mean most testing will underestimate his maximal abilities (though they do predict his performance in the settings we care about).

I suspect he is not unique.