Android存儲系統之源碼篇
/framework/base/services/java/com/android/server/SystemServer.java
/framework/base/services/core/java/com/android/server/MountService.java
/framework/base/services/core/java/com/android/server/NativeDaemonConnector.java
/framework/base/services/core/java/com/android/server/NativeDaemonEvent.java
/system/vold/Main.cpp
/system/vold/VolumeManager.cpp
/system/vold/NetlinkManager.cpp
/system/vold/NetlinkHandler.cpp
/system/vold/CommandListener.cpp
/system/vold/VoldCommand.cpp
/system/vold/VolumeBase.cpp
/system/vold/PublicVolume.cpp
/system/vold/EmulatedVolume.cpp
/system/vold/PublicVolume.cpp
/system/vold/Disk.cpp
/system/core/libsysutils/src/NetlinkListener.cpp
/system/core/libsysutils/src/SocketListener.cpp
/system/core/libsysutils/src/FrameworkListener.cpp
/system/core/libsysutils/src/FrameworkCommand.cpp
/system/core/include/sysutils/NetlinkListener.h
/system/core/include/sysutils/SocketListener.h
/system/core/include/sysutils/FrameworkListener.h
/system/core/include/sysutils/FrameworkCommand.h</code></pre>
一、概述
本文主要介紹跟存儲相關的模塊MountService和Vold,并不涉及底層文件系統。
- MountService:Android Binder服務,運行在system_server進程,用于跟Vold進行消息通信,比如 MountService 向 Vold 發送掛載SD卡的命令,或者接收到來自 Vold 的外設熱插拔事件。
- Vold(全稱為Volume Daemon),用于管理外部存儲設備的Native守護進程,這是一個非常重要的守護進程,由NetlinkManager,VolumeManager,CommandListener這3部分組成。
MountService的NativeDaemonConnector(Client端)和 Vold的CL模塊(Server端)建立socket通信。
- Linux Kernel:通過uevent向Vold的NetlinkManager發送Uevent事件;
- NetlinkManager:接收來自Kernel的 Uevent 事件,再轉發給VolumeManager;
- VolumeManager: 接收來自NetlinkManager的事件,再轉發給CommandListener進行處理;
- CommandListener: 接收來自VolumeManager的事件,通過 socket 通信方式發送給MountService;
- MountService: 接收來自CommandListener的事件。
1.1 存儲架構設計
存儲架構從進程/線程視角:
- Java層采用1個system_server主線程+3個子線程; 另外還會使用到系統進程中的兩個線程”android.fg”和”android.io”。
- Native采用1個vold主線程+3個子線程+1子進程的架構;
先看看Java層的線程:
root@gityuan:/ # ps -t | grep 1212
system 1212 557 2334024 160340 SyS_epoll_ 7faedddbe4 S system_server
system 2662 1212 2334024 160340 SyS_epoll_ 7faedddbe4 S MountService
system 2663 1212 2334024 160340 unix_strea 7faedde73c S VoldConnector
system 2664 1212 2334024 160340 unix_strea 7faedde73c S CryptdConnector
...
再看看Native層的線程:
root@gityuan:/ # ps -t | grep " 387 "
USER PID PPID VSIZE RSS WCHAN PC NAME
root 387 1 13572 2912 hrtimer_na 7fa34755d4 S /system/bin/vold
root 397 387 13572 2912 poll_sched 7fa3474d1c S vold
root 399 387 13572 2912 poll_sched 7fa3474d1c S vold
root 400 387 13572 2912 poll_sched 7fa3474d1c S vold
media_rw 2702 387 7140 2036 inotify_re 7f84b1d6ac S /system/bin/sdcard
小技巧:有讀者可能會好奇,為什么/system/bin/sdcard是子進程,而非子線程呢?要回答這個問題,有兩個方法,其一就是直接看擼源碼,會發現這是通過 fork 方式創建的,而其他子線程都是通過 pthread_create 方式創建的。當然其實還有個更快捷的小技巧,就是直接看上圖中的第4列,這一列的含義是 VSIZE ,代表的是進程虛擬地址空間大小,是否共享地址空間,這是進程與線程最大的區別,再來看看/sdcard的VSIZE大小跟父進程不一樣,基本可以確實/sdcard是子進程。
Tips: 同一個模塊可以運行在各個不同的進程/線程, 同一個進程可以運行不同模塊的代碼。
1.2 NativeDaemonEvent
響應碼
事件類別
對應方法
[100, 200)
部分響應,隨后繼續產生事件
isClassContinue
[200, 300)
成功響應
isClassOk
[400, 500)
遠程服務端錯誤
isClassServerError
[500, 600)
本地客戶端錯誤
isClassClientError
[600, 700)
遠程Vold進程自觸發的事件
isClassUnsolicited
例如當操作執行成功,VoldConnector線程能收到類似“RCV <- {200 3 Command succeeded}”的響應事件。
這里在詳細列舉遠程Vold進程的那些”不請自來”的事件,也就是指底層觸發的響應碼,范圍為[600,700)
命令
響應嗎
DISK_CREATED
640
DISK_SIZE_CHANGED
641
DISK_LABEL_CHANGED
642
DISK_SCANNED
643
DISK_SYS_PATH_CHANGED
644
DISK_DESTROYED
649
VOLUME_CREATED
650
VOLUME_STATE_CHANGED
651
VOLUME_FS_TYPE_CHANGED
652
VOLUME_FS_UUID_CHANGED
653
VOLUME_FS_LABEL_CHANGED
654
VOLUME_PATH_CHANGED
655
VOLUME_INTERNAL_PATH_CHANGED
656
VOLUME_DESTROYED
659
MOVE_STATUS
660
BENCHMARK_RESULT
661
TRIM_RESULT
662
這些命令主要是針對disk,volume的一系列操作,比如設備創建,狀態、路徑改變,以及文件類型、uid、標簽改變等事件都是底層直接觸發。
二、MountService
MountService運行在system_server進程,在系統啟動到階段PHASE_WAIT_FOR_DEFAULT_DISPLAY后,進入startOtherServices會啟動MountService.
2.1 啟動
[-> SystemServer.java]
private void startOtherServices() {
...
IMountService mountService = null;
//啟動MountService服務,【見小節2.2】
mSystemServiceManager.startService(MOUNT_SERVICE_CLASS);
//等價new IMountService.Stub.Proxy(),即獲取MountService的proxy對象
mountService = IMountService.Stub.asInterface(
ServiceManager.getService("mount"));
...
mActivityManagerService.systemReady(new Runnable() {
public void run() {
//啟動到階段550【見小節2.7】
mSystemServiceManager.startBootPhase(
SystemService.PHASE_ACTIVITY_MANAGER_READY);
...
});
}</code></pre>
NotificationManagerService依賴于MountService,比如media/usb通知事件,所以需要先啟動MountService。此處MOUNT_SERVICE_CLASS=”com.android.server.MountService$Lifecycle”
2.2 startService
mSystemServiceManager.startService(MOUNT_SERVICE_CLASS)主要完成3件事:
- 創建MOUNT_SERVICE_CLASS所指類的Lifecycle對象;
- 將該對象添加SystemServiceManager的 mServices 服務列表;
- 最后調用Lifecycle的onStart()方法,主要工作量這個過程,如下:
[-> MountService.java]
class MountService extends IMountService.Stub
implements INativeDaemonConnectorCallbacks, Watchdog.Monitor {
public static class Lifecycle extends SystemService {
public void onStart() {
//創建MountService對象【見小節2.3】
mMountService = new MountService(getContext());
//登記Binder服務
publishBinderService("mount", mMountService);
}
...
}
...
}
創建MountService對象,并向Binder服務的大管家ServiceManager登記,該服務名為“mount”,對應服務對象為mMountService。登記之后,其他地方當需要MountService的服務時便可以通過服務名來向ServiceManager來查詢具體的MountService服務。
2.3 MountService
public MountService(Context context) {
sSelf = this;
mContext = context;
//FgThread線程名為“"android.fg",創建IMountServiceListener回調方法【見小節2.4】
mCallbacks = new Callbacks(FgThread.get().getLooper());
//獲取PKMS的Client端對象
mPms = (PackageManagerService) ServiceManager.getService("package");
//創建“MountService”線程
HandlerThread hthread = new HandlerThread(TAG);
hthread.start();
mHandler = new MountServiceHandler(hthread.getLooper());
//IoThread線程名為"android.io",創建OBB操作的handler
mObbActionHandler = new ObbActionHandler(IoThread.get().getLooper());
File dataDir = Environment.getDataDirectory();
File systemDir = new File(dataDir, "system");
mLastMaintenanceFile = new File(systemDir, LAST_FSTRIM_FILE);
//判斷/data/system/last-fstrim文件,不存在則創建,存在則更新最后修改時間
if (!mLastMaintenanceFile.exists()) {
(new FileOutputStream(mLastMaintenanceFile)).close();
...
} else {
mLastMaintenance = mLastMaintenanceFile.lastModified();
}
...
//將MountServiceInternalImpl登記到sLocalServiceObjects
LocalServices.addService(MountServiceInternal.class, mMountServiceInternal);
//創建用于VoldConnector的NDC對象【見小節2.5】
mConnector = new NativeDaemonConnector(this, "vold", MAX_CONTAINERS * 2, VOLD_TAG, 25,
null);
mConnector.setDebug(true);
//創建線程名為"VoldConnector"的線程,用于跟vold通信【見小節2.6】
Thread thread = new Thread(mConnector, VOLD_TAG);
thread.start();
//創建用于CryptdConnector工作的NDC對象
mCryptConnector = new NativeDaemonConnector(this, "cryptd",
MAX_CONTAINERS * 2, CRYPTD_TAG, 25, null);
mCryptConnector.setDebug(true);
//創建線程名為"CryptdConnector"的線程,用于加密
Thread crypt_thread = new Thread(mCryptConnector, CRYPTD_TAG);
crypt_thread.start();
//注冊監聽用戶添加、刪除的廣播
final IntentFilter userFilter = new IntentFilter();
userFilter.addAction(Intent.ACTION_USER_ADDED);
userFilter.addAction(Intent.ACTION_USER_REMOVED);
mContext.registerReceiver(mUserReceiver, userFilter, null, mHandler);
//內部私有volume的路徑為/data,該volume通過dumpsys mount是不會顯示的
addInternalVolume();
//默認為false
if (WATCHDOG_ENABLE) {
Watchdog.getInstance().addMonitor(this);
}
}</code></pre>
主要功能依次是:
- FgThread線程名為”android.fg”,創建IMountServiceListener回調方法;
- 創建并啟動線程名為”MountService”的handlerThread;
- IoThread線程名為”android.io”,創建OBB操作的handler;
- 創建NativeDaemonConnector對象
- 創建并啟動線程名為”VoldConnector”的線程;
- 創建并啟動線程名為”CryptdConnector”的線程;
- 注冊監聽用戶添加、刪除的廣播;
該過程總共創建了3個線程:”MountService”,”VoldConnector”,”CryptdConnector”,另外還會使用到系統進程中的兩個線程”android.fg”和”android.io”
2.4 Callbacks
class MountService {
...
private static class Callbacks extends Handler {
private final RemoteCallbackList<IMountServiceListener>
mCallbacks = new RemoteCallbackList<>();
public Callbacks(Looper looper) {
super(looper);
}
...
}
}
創建Callbacks時的Looper為FgThread.get().getLooper(),其中 FgThread 采用單例模式,是一個線程名為”android.fg”的HandlerThread。另外,Callbacks對象有一個成員變量 mCallbacks ,如下:
[-> RemoteCallbackList.java]
public class RemoteCallbackList<E extends IInterface> {
ArrayMap<IBinder, Callback> mCallbacks
= new ArrayMap<IBinder, Callback>();
//Binder死亡通知
private final class Callback implements IBinder.DeathRecipient {
public void binderDied() {
...
}
}
//注冊死亡回調
public boolean register(E callback, Object cookie) {
synchronized (mCallbacks) {
...
IBinder binder = callback.asBinder();
Callback cb = new Callback(callback, cookie);
binder.linkToDeath(cb, 0);
mCallbacks.put(binder, cb);
...
}
}
...
}</code></pre>
通過 register() 方法添加IMountServiceListener對象信息到 mCallbacks 成員變量。RemoteCallbackList的內部類Callback繼承于IBinder.DeathRecipient,很顯然這是死亡通知,當binder服務端進程死亡后,回調binderDied方法通知binder客戶端進行相應地處理。
2.5 NativeDaemonConnector
[-> NativeDaemonConnector.java]
NativeDaemonConnector(INativeDaemonConnectorCallbacks callbacks, String socket,
int responseQueueSize, String logTag, int maxLogSize, PowerManager.WakeLock wl) {
this(callbacks, socket, responseQueueSize, logTag, maxLogSize, wl,
FgThread.get().getLooper());
}
NativeDaemonConnector(INativeDaemonConnectorCallbacks callbacks, String socket,
int responseQueueSize, String logTag, int maxLogSize, PowerManager.WakeLock wl,
Looper looper) {
mCallbacks = callbacks;
//socket名為"vold"
mSocket = socket;
//對象響應個數為500
mResponseQueue = new ResponseQueue(responseQueueSize);
mWakeLock = wl;
if (mWakeLock != null) {
mWakeLock.setReferenceCounted(true);
}
mLooper = looper;
mSequenceNumber = new AtomicInteger(0);
//TAG為"VoldConnector"
TAG = logTag != null ? logTag : "NativeDaemonConnector";
mLocalLog = new LocalLog(maxLogSize);
}</code></pre>
- mLooper為FgThread.get().getLooper(),即運行在”android.fg”線程;
- mResponseQueue對象中成員變量 mPendingCmds 數據類型為LinkedList,記錄著vold進程上報的響應事件,事件個數上限為500。
2.6 NDC.run
[-> NativeDaemonConnector.java]
final class NativeDaemonConnector implements Runnable, Handler.Callback, Watchdog.Monitor {
public void run() {
mCallbackHandler = new Handler(mLooper, this);
while (true) {
try {
//監聽vold的socket【見小節2.7】
listenToSocket();
} catch (Exception e) {
loge("Error in NativeDaemonConnector: " + e);
SystemClock.sleep(5000);
}
}
}
}</code></pre>
“VoldConnector”線程建立了”vold“的socket的客戶端,通過循環方式不斷監聽Vold服務端發送過來的消息。 另外,同理還有一個線程“CryptdConnector”也采用類似的方式,建立了“cryptd”的socket客戶端,監聽Vold中另個線程發送過來的消息。
MountService與NDC都啟動,那么接下來到系統啟動到達階段PHASE_ACTIVITY_MANAGER_READY,則調用到onBootPhase方法。
2.7 onBootPhase
[-> MountService.java ::Lifecycle]
由于MountService的內部Lifecycle已添加SystemServiceManager的 mServices 服務列表;故到系統啟動到 PHASE_ACTIVITY_MANAGER_READY 時會回調 mServices 中的 onBootPhase 方法
public static class Lifecycle extends SystemService {
public void onBootPhase(int phase) {
if (phase == SystemService.PHASE_ACTIVITY_MANAGER_READY) {
mMountService.systemReady();
}
}
}
再調用MountService.systemReady方法,該方法主要是通過mHandler發送消息。
private void systemReady() {
mSystemReady = true;
mHandler.obtainMessage(H_SYSTEM_READY).sendToTarget();
}
此處mHandler = new MountServiceHandler(hthread.getLooper())可知采用的是線程”MountService”中的Looper。到此system_server主線程通過handler向線程”MountService”發送H_SYSTEM_READY消息,接下來進入線程”MountService”的MountServiceHandler對象(簡稱MSH)的handleMessage()來處理相關的消息。
2.8 MSH.handleMessage
[-> MountService.java ::MountServiceHandler]
class MountServiceHandler extends Handler {
public void handleMessage(Message msg) {
switch (msg.what) {
case H_SYSTEM_READY: {
handleSystemReady(); //【見小節2.9】
break;
}
...
}
}
}
2.9 handleSystemReady
[-> MountService.java]
private void handleSystemReady() {
synchronized (mLock) {
//【見小節2.10】
resetIfReadyAndConnectedLocked();
}
//計劃執行日常的fstrim操作【】
MountServiceIdler.scheduleIdlePass(mContext);
}</code></pre>
2.10 resetIfReadyAndConnectedLocked
[-> MountService.java]
private void resetIfReadyAndConnectedLocked() {
Slog.d(TAG, "Thinking about reset, mSystemReady=" + mSystemReady
+ ", mDaemonConnected=" + mDaemonConnected);
//當系統啟動到階段550,并且已經與vold守護進程建立連接,則執行reset
if (mSystemReady && mDaemonConnected) {
killMediaProvider();
mDisks.clear();
mVolumes.clear();
//將/data為路徑的private volume添加到mVolumes
addInternalVolume();
try {
//【見小節2.11】
mConnector.execute("volume", "reset");
//告知所有已經存在和啟動的users
final UserManager um = mContext.getSystemService(UserManager.class);
final List<UserInfo> users = um.getUsers();
for (UserInfo user : users) {
mConnector.execute("volume", "user_added", user.id, user.serialNumber);
}
for (int userId : mStartedUsers) {
mConnector.execute("volume", "user_started", userId);
}
} catch (NativeDaemonConnectorException e) {
Slog.w(TAG, "Failed to reset vold", e);
}
}
}</code></pre>
2.11 NDC.execute
[-> NativeDaemonConnector.java]
public NativeDaemonEvent execute(String cmd, Object... args)
throws NativeDaemonConnectorException {
return execute(DEFAULT_TIMEOUT, cmd, args);
}
其中DEFAULT_TIMEOUT等于1分鐘,即命令執行超時時長為1分鐘。經過層層調用,executeForList()
public NativeDaemonEvent[] executeForList(long timeoutMs, String cmd, Object... args)
throws NativeDaemonConnectorException {
final long startTime = SystemClock.elapsedRealtime();
final ArrayList<NativeDaemonEvent> events = Lists.newArrayList();
final StringBuilder rawBuilder = new StringBuilder();
final StringBuilder logBuilder = new StringBuilder();
//mSequenceNumber初始化值為0,每執行一次該方法則進行加1操作
final int sequenceNumber = mSequenceNumber.incrementAndGet();
makeCommand(rawBuilder, logBuilder, sequenceNumber, cmd, args);
//例如:“3 volume reset”
final String rawCmd = rawBuilder.toString();
final String logCmd = logBuilder.toString();
log("SND -> {" + logCmd + "}");
synchronized (mDaemonLock) {
//將cmd寫入到socket的輸出流
mOutputStream.write(rawCmd.getBytes(StandardCharsets.UTF_8));
...
}
NativeDaemonEvent event = null;
do {
//【見小節2.12】
event = mResponseQueue.remove(sequenceNumber, timeoutMs, logCmd);
events.add(event);
//當收到的事件響應碼屬于[100,200)區間,則繼續等待后續事件上報
} while (event.isClassContinue());
final long endTime = SystemClock.elapsedRealtime();
//對于執行時間超過500ms則會記錄到log
if (endTime - startTime > WARN_EXECUTE_DELAY_MS) {
loge("NDC Command {" + logCmd + "} took too long (" + (endTime - startTime) + "ms)");
}
...
return events.toArray(new NativeDaemonEvent[events.size()]);
}</code></pre>
首先,將帶執行的命令mSequenceNumber執行加1操作,再將cmd(例如 3 volume reset )寫入到socket的輸出流,通過循環與poll機制等待執行底層響應該操作結果,否則直到1分鐘超時才結束該方法。即便收到底層的響應碼,如果響應碼屬于[100,200)區間,則繼續阻塞等待后續事件上報。
2.12 ResponseQueue.remove
[-> MountService.java ::ResponseQueue]
private static class ResponseQueue {
public NativeDaemonEvent remove(int cmdNum, long timeoutMs, String logCmd) {
PendingCmd found = null;
synchronized (mPendingCmds) {
//從mPendingCmds查詢cmdNum
for (PendingCmd pendingCmd : mPendingCmds) {
if (pendingCmd.cmdNum == cmdNum) {
found = pendingCmd;
break;
}
}
//如果已有的mPendingCmds中查詢不到,則創建一個新的PendingCmd
if (found == null) {
found = new PendingCmd(cmdNum, logCmd);
mPendingCmds.add(found);
}
found.availableResponseCount--;
if (found.availableResponseCount == 0) mPendingCmds.remove(found);
}
NativeDaemonEvent result = null;
try {
//采用poll輪詢方式等待底層上報該事件,直到1分鐘超時
result = found.responses.poll(timeoutMs, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {}
return result;
}
}
mPendingCmds中的內容是如何添加的呢?其實是在NDC.listenToSocket循環監聽到消息時添加的,則接下來看看監聽過程。 這里用到poll,先來看看 responses = new ArrayBlockingQueue<NativeDaemonEvent>(10) ,這是一個長度為10的可阻塞隊列。 這里的poll也是阻塞的方式來輪詢事件。
responses.poll
[-> ArrayBlockingQueue.java]
public E poll(long timeout, TimeUnit unit) throws InterruptedException {
long nanos = unit.toNanos(timeout);
final ReentrantLock lock = this.lock;
//可中斷的鎖等待
lock.lockInterruptibly();
try {
//當隊列長度為空,循環等待
while (count == 0) {
if (nanos <= 0)
return null;
nanos = notEmpty.awaitNanos(nanos);
}
return extract();
} finally {
lock.unlock();
}
}
小知識:這里用到了ReentrantLock同步鎖,該鎖跟synchronized有功能有很相似,用于多線程并發訪問。那么ReentrantLock與synchronized相比,ReentrantLock優勢:
- ReentrantLock功能更為強大,比如有時間鎖等候,可中斷鎖等候(lockInterruptibly),鎖投票等功能;
- ReentrantLock性能更好些;
- ReentrantLock提供可輪詢的鎖請求(tryLock),相對不容易產生死鎖;而synchronized只要進入,要么成功獲取,要么一直阻塞等待。
ReentrantLock的劣勢:
- lock必須在finally塊顯式地釋放,否則如果代碼拋出Exception,鎖將一直得不到釋放;對于synchronized而言,JVM或者ART虛擬機都會確保該鎖能自動釋放。
- synchronized鎖,在dump線程轉儲時會記錄鎖信息,對于分析調試大有裨益;對于Lock來說,只是普通類,虛擬機無法識別。
2.13 listenToSocket
[-> NativeDaemonConnector.java]
private void listenToSocket() throws IOException {
LocalSocket socket = null;
try {
socket = new LocalSocket();
LocalSocketAddress address = determineSocketAddress();
//建立與"/dev/socket/vold"的socket連接
socket.connect(address);
InputStream inputStream = socket.getInputStream();
synchronized (mDaemonLock) {
mOutputStream = socket.getOutputStream();
}
//建立連接后,回調MS.onDaemonConnected【見小節2.15】
mCallbacks.onDaemonConnected();
byte[] buffer = new byte[BUFFER_SIZE];
int start = 0;
while (true) {
int count = inputStream.read(buffer, start, BUFFER_SIZE - start);
...
for (int i = 0; i < count; i++) {
if (buffer[i] == 0) {
final String rawEvent = new String(
buffer, start, i - start, StandardCharsets.UTF_8);
boolean releaseWl = false;
try {
//解析socket服務端發送的event
final NativeDaemonEvent event = NativeDaemonEvent.parseRawEvent(
rawEvent);
log("RCV <- {" + event + "}");
//當事件的響應碼區間為[600,700),則發送消息交由mCallbackHandler處理
if (event.isClassUnsolicited()) {
if (mCallbacks.onCheckHoldWakeLock(event.getCode())
&& mWakeLock != null) {
mWakeLock.acquire();
releaseWl = true;
}
if (mCallbackHandler.sendMessage(mCallbackHandler.obtainMessage(
event.getCode(), event.getRawEvent()))) {
releaseWl = false;
}
} else {
//對于其他的響應碼則添加到mResponseQueue隊列【見小節2.14】
mResponseQueue.add(event.getCmdNumber(), event);
}
} catch (IllegalArgumentException e) {
log("Problem parsing message " + e);
} finally {
if (releaseWl) {
mWakeLock.acquire();
}
}
start = i + 1;
}
}
...
}
} catch (IOException ex) {
throw ex;
} finally {
//收尾清理類工作,關閉mOutputStream, socket
...
}
}</code></pre>
這里有一個動作是mResponseQueue.add(),通過該方法便能觸發ResponseQueue.poll阻塞操作繼續往下執行。
2.14 ResponseQueue.add
[-> NativeDaemonConnector.java]
private static class ResponseQueue {
public void add(int cmdNum, NativeDaemonEvent response) {
PendingCmd found = null;
synchronized (mPendingCmds) {
for (PendingCmd pendingCmd : mPendingCmds) {
if (pendingCmd.cmdNum == cmdNum) {
found = pendingCmd;
break;
}
}
//沒有找到則創建相應的PendingCmd
if (found == null) {
while (mPendingCmds.size() >= mMaxCount) {
PendingCmd pendingCmd = mPendingCmds.remove();
}
found = new PendingCmd(cmdNum, null);
mPendingCmds.add(found);
}
found.availableResponseCount++;
if (found.availableResponseCount == 0) mPendingCmds.remove(found);
}
try {
found.responses.put(response);
} catch (InterruptedException e) { }
}
}
responses.put
[-> ArrayBlockingQueue.java]
public void put(E e) throws InterruptedException {
checkNotNull(e);
final ReentrantLock lock = this.lock;
lock.lockInterruptibly();
try {
//當隊列滿了則等待
while (count == items.length)
notFull.await();
insert(e);
} finally {
lock.unlock();
}
}
2.15 MS.onDaemonConnected
[-> MountService.java]
public void onDaemonConnected() {
mDaemonConnected = true;
mHandler.obtainMessage(H_DAEMON_CONNECTED).sendToTarget();
}
當前主線程發送消息H_DAEMON_CONNECTED給線程“MountService”,該線程收到消息后調用MountServiceHandler的handleMessage()相應分支后,進而調用handleDaemonConnected()方法。
private void handleDaemonConnected() {
synchronized (mLock) {
resetIfReadyAndConnectedLocked();
}
//類型為CountDownLatch,用于多線程同步,阻塞await()直到計數器為零
mConnectedSignal.countDown();
if (mConnectedSignal.getCount() != 0) {
return;
}
//調用PMS來加載ASECs
mPms.scanAvailableAsecs();
//用于通知ASEC掃描已完成
mAsecsScanned.countDown();
}</code></pre>
這里的PMS.scanAvailableAsecs()經過層層調用,最終核心工作還是通過MountService.getSecureContainerList。
[-> MountService.java]
public String[] getSecureContainerList() {
enforcePermission(android.Manifest.permission.ASEC_ACCESS);
//等待mConnectedSignal計數鎖達到零
waitForReady();
//當沒有掛載Primary卷設備,則彈出警告
warnOnNotMounted();
try {
//向vold進程發送asec list命令
return NativeDaemonEvent.filterMessageList(
mConnector.executeForList("asec", "list"), VoldResponseCode.AsecListResult);
} catch (NativeDaemonConnectorException e) {
return new String[0];
}
}</code></pre>
三、Vold
Vold是由開機過程中解析init.rc時啟動:
on post-fs-data
start vold
Vold的service定義如下:
service vold /system/bin/vold
class core
socket vold stream 0660 root mount
socket cryptd stream 0660 root mount
ioprio be 2
接下來便進入main()方法:
3.1 main
[-> system/vold/Main.cpp]
int main(int argc, char* argv) {
setenv("ANDROID_LOG_TAGS", ":v", 1);
android::base::InitLogging(argv, android::base::LogdLogger(android::base::SYSTEM));
VolumeManager *vm;
CommandListener *cl;
CryptCommandListener *ccl;
NetlinkManager *nm;
//解析參數,設置contenxt
parse_args(argc, argv);
...
fcntl(android_get_control_socket("vold"), F_SETFD, FD_CLOEXEC);
fcntl(android_get_control_socket("cryptd"), F_SETFD, FD_CLOEXEC);
mkdir("/dev/block/vold", 0755);
//用于cryptfs檢查,并mount加密的文件系統
klog_set_level(6);
//創建單例對象VolumeManager 【見小節3.2.1】
if (!(vm = VolumeManager::Instance())) {
exit(1);
}
//創建單例對象NetlinkManager 【見小節3.3.1】
if (!(nm = NetlinkManager::Instance())) {
exit(1);
}
if (property_get_bool("vold.debug", false)) {
vm->setDebug(true);
}
// 創建CommandListener對象 【見小節3.4.1】
cl = new CommandListener();
// 創建CryptCommandListener對象 【見小節3.5.1】
ccl = new CryptCommandListener();
//【見小節3.2.2】
vm->setBroadcaster((SocketListener *) cl);
//【見小節3.3.2】
nm->setBroadcaster((SocketListener *) cl);
if (vm->start()) { //【見小節3.2.3】
exit(1);
}
process_config(vm); //【見小節3.2.4】
if (nm->start()) { //【見小節3.3.3】
exit(1);
}
coldboot("/sys/block");
//啟動響應命令的監聽器 //【見小節3.4.2】
if (cl->startListener()) {
exit(1);
}
if (ccl->startListener()) {
exit(1);
}
//Vold成為監聽線程
while(1) {
sleep(1000);
}
exit(0);
}</code></pre>
該方法的主要功能是創建下面4個對象并啟動
- VolumeManager
- NetlinkManager (NetlinkHandler)
- CommandListener
- CryptCommandListener
接下來分別說說幾個類:
3.2 VolumeManager
3.2.1 創建
[-> VolumeManager.cpp]
VolumeManager *VolumeManager::Instance() {
if (!sInstance)
sInstance = new VolumeManager();
return sInstance;
}
創建單例模式的VolumeManager對象
VolumeManager::VolumeManager() {
mDebug = false;
mActiveContainers = new AsecIdCollection();
mBroadcaster = NULL;
mUmsSharingCount = 0;
mSavedDirtyRatio = -1;
//當UMS獲取時,則設置dirty ratio為0
mUmsDirtyRatio = 0;
}
3.2.2 vm->setBroadcaster
void setBroadcaster(SocketListener *sl) {
mBroadcaster = sl;
}
將新創建的 CommandListener 對象sl賦值給vm對象的成員變量 mBroadcaster
3.2.3 vm->start
int VolumeManager::start() {
//卸載所有設備,以提供最干凈的環境
unmountAll();
//創建Emulated內部存儲
mInternalEmulated = std::shared_ptr<android::vold::VolumeBase>(
new android::vold::EmulatedVolume("/data/media"));
mInternalEmulated->create();
return 0;
}</code></pre>
mInternalEmulated的據類型為 EmulatedVolume ,設備路徑為 /data/media ,id和label為“emulated”,mMountFlags=0。 EmulatedVolume 繼承于 VolumeBase
3.2.3.1 unmountAll
int VolumeManager::unmountAll() {
std::lock_guard<std::mutex> lock(mLock);
//卸載內部存儲
if (mInternalEmulated != nullptr) {
mInternalEmulated->unmount();
}
//卸載外部存儲
for (auto disk : mDisks) {
disk->unmountAll();
}
FILE* fp = setmntent("/proc/mounts", "r");
if (fp == NULL) {
return -errno;
}
std::list<std::string> toUnmount;
mntent* mentry;
while ((mentry = getmntent(fp)) != NULL) {
if (strncmp(mentry->mnt_dir, "/mnt/", 5) == 0
|| strncmp(mentry->mnt_dir, "/storage/", 9) == 0) {
toUnmount.push_front(std::string(mentry->mnt_dir));
}
}
endmntent(fp);
for (auto path : toUnmount) {
//強制卸載
android::vold::ForceUnmount(path);
}
return 0;
}</code></pre>
此處打開的”/proc/mounts”每一行內容依次是文件名,目錄,類型,操作。例如:
/dev/fuse /mnt/runtime/default/emulated fuse rw,nosuid,nodev,...
該方法的主要工作是卸載:
- 內部存儲mInternalEmulated;
- 外部存儲mDisks,比如sdcard等;
- “/proc/mounts”中目錄包含mnt或者storage的路徑;
卸載內部存儲:
status_t EmulatedVolume::doUnmount() {
if (mFusePid > 0) {
kill(mFusePid, SIGTERM);
TEMP_FAILURE_RETRY(waitpid(mFusePid, nullptr, 0));
mFusePid = 0;
}
KillProcessesUsingPath(getPath());
//強制卸載fuse路徑
ForceUnmount(mFuseDefault);
ForceUnmount(mFuseRead);
ForceUnmount(mFuseWrite);
rmdir(mFuseDefault.c_str());
rmdir(mFuseRead.c_str());
rmdir(mFuseWrite.c_str());
mFuseDefault.clear();
mFuseRead.clear();
mFuseWrite.clear();
return OK;
}</code></pre>
KillProcessesUsingPath 的功能很強大,通過文件path來查看其所在進程,并殺掉相應進程。當以下5處任意一處存在與path相同的地方,則會殺掉相應的進程:
- proc/<pid>/fd ,打開文件;
- proc/<pid>/maps 打開文件映射;
- proc/<pid>/cwd 鏈接文件;
- proc/<pid>/root 鏈接文件;
- proc/<pid>/exe 鏈接文件;
3.2.3.2 EV.create
[-> VolumeBase.cpp]
status_t VolumeBase::create() {
mCreated = true;
status_t res = doCreate();
//通知VolumeCreated事件
notifyEvent(ResponseCode::VolumeCreated,
StringPrintf("%d \"%s\" \"%s\"", mType, mDiskId.c_str(), mPartGuid.c_str()));
//設置為非掛載狀態
setState(State::kUnmounted);
return res;
}
void VolumeBase::notifyEvent(int event, const std::string& value) {
if (mSilent) return;
//通過socket向MountService發送創建volume的命令(650)
VolumeManager::Instance()->getBroadcaster()->sendBroadcast(event,
StringPrintf("%s %s", getId().c_str(), value.c_str()).c_str(), false);
}</code></pre>
3.2.4 process_config(vm)
[-> system/vold/Main.cpp]
static int process_config(VolumeManager *vm) {
//獲取Fstab路徑
std::string path(android::vold::DefaultFstabPath());
fstab = fs_mgr_read_fstab(path.c_str());
...
bool has_adoptable = false;
for (int i = 0; i < fstab->num_entries; i++) {
if (fs_mgr_is_voldmanaged(&fstab->recs[i])) {
if (fs_mgr_is_nonremovable(&fstab->recs[i])) {
LOG(WARNING) << "nonremovable no longer supported; ignoring volume";
continue;
}
std::string sysPattern(fstab->recs[i].blk_device);
std::string nickname(fstab->recs[i].label);
int flags = 0;
if (fs_mgr_is_encryptable(&fstab->recs[i])) {
flags |= android::vold::Disk::Flags::kAdoptable;
has_adoptable = true;
}
if (fs_mgr_is_noemulatedsd(&fstab->recs[i])
|| property_get_bool("vold.debug.default_primary", false)) {
flags |= android::vold::Disk::Flags::kDefaultPrimary;
}
vm->addDiskSource(std::shared_ptr<VolumeManager::DiskSource>(
new VolumeManager::DiskSource(sysPattern, nickname, flags)));
}
}
property_set("vold.has_adoptable", has_adoptable ? "1" : "0");
return 0;
}</code></pre>
Fstab路徑:首先通過 getprop ro.hardware ,比如高通芯片則為 qcom 那么Fstab路徑就是 /fstab.qcom ,那么該文件的具體內容,例如(當然這個不同手機會有所不同):
src
mnt_point
type
mnt_flags and options
fs_mgr_flags
/dev/block/bootdevice/by-name/system
/system
ext4
ro,barrier=1,discard
wait,verify
/dev/block/bootdevice/by-name/userdata
/data
ext4
nosuid,nodev,barrier=1,noauto_da_alloc,discard
wait,check,forceencrypt=footer
/dev/block/bootdevice/by-name/cust
/cust
ext4
nosuid,nodev,barrier=1
wait,check
/devices/soc.0/7864900.sdhci/mmc_host*
/storage/sdcard1
vfat
nosuid,nodev
wait,voldmanaged=sdcard1:auto,noemulatedsd,encryptable=footer
/dev/block/bootdevice/by-name/config
/frp
emmc
defaults defaults
/devices/platform/msm_hsusb*
/storage/usbotg
vfat
nosuid,nodev
wait,voldmanaged=usbotg:auto,encryptable=footer
3.3 NetlinkManager
3.3.1 創建
[-> NetlinkManager.cpp]
NetlinkManager *NetlinkManager::Instance() {
if (!sInstance)
sInstance = new NetlinkManager();
return sInstance;
}
3.3.2 nm->setBroadcaster
void setBroadcaster(SocketListener *sl) {
mBroadcaster = sl;
}
3.3.3 nm->start
int NetlinkManager::start() {
struct sockaddr_nl nladdr;
int sz = 64 * 1024;
int on = 1;
memset(&nladdr, 0, sizeof(nladdr));
nladdr.nl_family = AF_NETLINK;
nladdr.nl_pid = getpid(); //記錄當前進程的pid
nladdr.nl_groups = 0xffffffff;
//創建event socket
if ((mSock = socket(PF_NETLINK, SOCK_DGRAM | SOCK_CLOEXEC,
NETLINK_KOBJECT_UEVENT)) < 0) {
return -1;
}
//設置uevent的SO_RCVBUFFORCE選項
if (setsockopt(mSock, SOL_SOCKET, SO_RCVBUFFORCE, &sz, sizeof(sz)) < 0) {
goto out;
}
//設置uevent的SO_PASSCRED選項
if (setsockopt(mSock, SOL_SOCKET, SO_PASSCRED, &on, sizeof(on)) < 0) {
goto out;
}
//綁定uevent socket
if (bind(mSock, (struct sockaddr *) &nladdr, sizeof(nladdr)) < 0) {
goto out;
}
//創建NetlinkHandler【見小節3.3.4】
mHandler = new NetlinkHandler(mSock);
//啟動NetlinkHandler【見小節3.3.5】
if (mHandler->start()) {
goto out;
}
return 0;
out:
close(mSock);
return -1;
}</code></pre>
3.3.4 NetlinkHandler
NetlinkHandler繼承于 NetlinkListener , NetlinkListener 繼承于 SocketListener 。new NetlinkHandler(mSock)中參數mSock是用于與Kernel進行通信的socket對象。由于這個繼承關系,當NetlinkHandler初始化時會調用基類的初始化,最終調用到:
[-> SocketListener.cpp]
SocketListener::SocketListener(int socketFd, bool listen) {
//listen=false
init(NULL, socketFd, listen, false);
}
void SocketListener::init(const char *socketName, int socketFd, bool listen, bool useCmdNum) {
mListen = listen;
mSocketName = socketName;
//用于監聽Kernel發送過程的uevent事件
mSock = socketFd;
mUseCmdNum = useCmdNum;
//初始化同步鎖
pthread_mutex_init(&mClientsLock, NULL);
//創建socket通信的client端
mClients = new SocketClientCollection();
}</code></pre>
到此,mListen = false; mSocketName = NULL; mUseCmdNum = false。 另外,這里用到的同步鎖,用于控制多線程并發訪問。 接著在來看看start過程:
3.3.5 NH->start
[-> NetlinkHandler.cpp]
int NetlinkHandler::start() {
return this->startListener();
}
[-> SocketListener.cpp]
int SocketListener::startListener() {
return startListener(4);
}
int SocketListener::startListener(int backlog) {
...
//mListen =false
if (mListen && listen(mSock, backlog) < 0) {
return -1;
} else if (!mListen)
//創建SocketClient對象,并加入到mClients隊列
mClients->push_back(new SocketClient(mSock, false, mUseCmdNum));
//創建匿名管道
if (pipe(mCtrlPipe)) {
return -1;
}
//創建工作線程,線程運行函數threadStart【見小節3.3.6】
if (pthread_create(&mThread, NULL, SocketListener::threadStart, this)) {
return -1;
}
return 0;
}</code></pre>
mCtrlPipe是匿名管道,這是一個二元數組,mCtrlPipe[0]從管道讀數據,mCtrlPipe[1]從管道寫數據。
3.3.6 threadStart
[-> SocketListener.cpp]
void *SocketListener::threadStart(void *obj) {
SocketListener *me = reinterpret_cast<SocketListener *>(obj);
//【見小節3.3.7】
me->runListener();
pthread_exit(NULL); //線程退出
return NULL;
}
3.3.7 SL->runListener
[-> SocketListener.cpp]
void SocketListener::runListener() {
SocketClientCollection pendingList;
while(1) {
SocketClientCollection::iterator it;
fd_set read_fds;
int rc = 0;
int max = -1;
FD_ZERO(&read_fds);
if (mListen) {
max = mSock;
FD_SET(mSock, &read_fds);
}
FD_SET(mCtrlPipe[0], &read_fds);
if (mCtrlPipe[0] > max)
max = mCtrlPipe[0];
pthread_mutex_lock(&mClientsLock);
for (it = mClients->begin(); it != mClients->end(); ++it) {
// NB: calling out to an other object with mClientsLock held (safe)
int fd = (*it)->getSocket();
FD_SET(fd, &read_fds);
if (fd > max) {
max = fd;
}
}
pthread_mutex_unlock(&mClientsLock);
SLOGV("mListen=%d, max=%d, mSocketName=%s", mListen, max, mSocketName);
if ((rc = select(max + 1, &read_fds, NULL, NULL, NULL)) < 0) {
if (errno == EINTR)
continue;
SLOGE("select failed (%s) mListen=%d, max=%d", strerror(errno), mListen, max);
sleep(1);
continue;
} else if (!rc)
continue;
if (FD_ISSET(mCtrlPipe[0], &read_fds)) {
char c = CtrlPipe_Shutdown;
TEMP_FAILURE_RETRY(read(mCtrlPipe[0], &c, 1));
if (c == CtrlPipe_Shutdown) {
break;
}
continue;
}
if (mListen && FD_ISSET(mSock, &read_fds)) {
struct sockaddr addr;
socklen_t alen;
int c;
do {
alen = sizeof(addr);
c = accept(mSock, &addr, &alen);
SLOGV("%s got %d from accept", mSocketName, c);
} while (c < 0 && errno == EINTR);
if (c < 0) {
SLOGE("accept failed (%s)", strerror(errno));
sleep(1);
continue;
}
fcntl(c, F_SETFD, FD_CLOEXEC);
pthread_mutex_lock(&mClientsLock);
mClients->push_back(new SocketClient(c, true, mUseCmdNum));
pthread_mutex_unlock(&mClientsLock);
}
/* Add all active clients to the pending list first */
pendingList.clear();
pthread_mutex_lock(&mClientsLock);
for (it = mClients->begin(); it != mClients->end(); ++it) {
SocketClient* c = *it;
// NB: calling out to an other object with mClientsLock held (safe)
int fd = c->getSocket();
if (FD_ISSET(fd, &read_fds)) {
pendingList.push_back(c);
c->incRef();
}
}
pthread_mutex_unlock(&mClientsLock);
/* Process the pending list, since it is owned by the thread,
* there is no need to lock it */
while (!pendingList.empty()) {
/* Pop the first item from the list */
it = pendingList.begin();
SocketClient* c = *it;
pendingList.erase(it);
/* Process it, if false is returned, remove from list */
if (!onDataAvailable(c)) {
release(c, false);
}
c->decRef();
}
}
}</code></pre>
3.4 CommandListener
3.4.1 創建
[-> CommandListener.cpp]
CommandListener::CommandListener() :
FrameworkListener("vold", true) {
registerCmd(new DumpCmd());
registerCmd(new VolumeCmd());
registerCmd(new AsecCmd());
registerCmd(new ObbCmd());
registerCmd(new StorageCmd());
registerCmd(new FstrimCmd());
}
3.4.1.1 FrameworkListener
[-> FrameworkListener.cpp]
FrameworkListener::FrameworkListener(const char *socketName, bool withSeq) :
SocketListener(socketName, true, withSeq) {
init(socketName, withSeq);
}
void FrameworkListener::init(const char socketName UNUSED, bool withSeq) {
mCommands = new FrameworkCommandCollection();
errorRate = 0;
mCommandCount = 0;
mWithSeq = withSeq; //true
}</code></pre>
3.4.1.2 SocketListener
[-> SocketListener.cpp]
SocketListener::SocketListener(const char
socketName, bool listen, bool useCmdNum) {
init(socketName, -1, listen, useCmdNum);
}
void SocketListener::init(const char socketName, int socketFd, bool listen, bool useCmdNum) {
mListen = listen; //true
mSocketName = socketName; //"vold"
mSock = socketFd; // -1
mUseCmdNum = useCmdNum; //true
pthread_mutex_init(&mClientsLock, NULL);
mClients = new SocketClientCollection();
}</code></pre>
socket名為“vold”
3.4.1.3 registerCmd
void FrameworkListener::registerCmd(FrameworkCommand
cmd) {
mCommands->push_back(cmd);
}
CommandListener::VolumeCmd::VolumeCmd() :
VoldCommand("volume") {
}</code></pre>
創建這些對象 DumpCmd,VolumeCmd,AsecCmd,ObbCmd,StorageCmd,FstrimCmd,并都加入到mCommands隊列。
3.4.2 cl->startListener
int SocketListener::startListener() {
return startListener(4);
}
int SocketListener::startListener(int backlog) {
if (!mSocketName && mSock == -1) {
...
} else if (mSocketName) {
//獲取“vold”所對應的句柄
if ((mSock = android_get_control_socket(mSocketName)) < 0) {
return -1;
}
fcntl(mSock, F_SETFD, FD_CLOEXEC);
}
//CL開始監聽
if (mListen && listen(mSock, backlog) < 0) {
return -1;
}
...
//創建匿名管道
if (pipe(mCtrlPipe)) {
return -1;
}
//創建工作線程
if (pthread_create(&mThread, NULL, SocketListener::threadStart, this)) {
return -1;
}
return 0;
}</code></pre>
四、 案例分析
MountService根據收到消息會發送VM處理。 例如待SD卡插入后,VM會將(來自NM的“Disk Insert”的)消息發送給MountService,而后MountService則發送“Mount”指令給Vold,指示它掛載這個SD卡。有些應用程序需要檢測外部存儲卡的插入/拔出事件,這些事件是由MountService通過Intent廣播發出的,例如外部存儲卡插入后,MountService就會發送ACTION_MEDIA_MOUNTED消息。
4.1 Kernel上報處理流程
插入SD卡動作: 通過硬件驅動會引起Kernel向NM發送uevent,NM轉發消息給VM,再通過CL發送給MountService;MountService收到后則向Vold發送Mount命令,來掛載SD卡。
整個流程: Kernel Uevent -> volumeManager -> NetlinkManager -> CommandListener -> NativeDaemonConnector -> MountService
Kernel發出Uevnt事件 SocketListener::startListener SocketListener::threadStart SocketListener::runListener NetlinkListener.onDataAvailable NetlinkHandler.onEvent VM.handleBlockEvent Disk.create Disk::notifyEvent SocketListener::sendBroadcast 發送socket事件給上層
4.2 framework下發處理流程
MountServic發送socket,執行mount SocketListener::startListener SocketListener::threadStart SocketListener::runListener FrameworkListener::onDataAvailable FrameworkListener::dispatchCommand VolumeCmd.runCommand VolumeBase.mount EmulatedVolume.doMount(內置) PublicVolume.doMount(外置) vfat::Check vfat::Mount fork (/sdcard)
4.3 startUser流程
AMS.systemReady goingCallback.run(); onBootPhase(550) MountService.systemReady(); => mSystemReady = true MountService.resetIfReadyAndConnectedLocked mConnector.execute(“volume”, “reset”); RCV <- {650 emulated –> VOLUME_CREATED RCV <- {650 public –> VOLUME_CREATED mConnector.execute(“volume”, “user_added”, user.id, user.serialNumber); mConnector.execute(“volume”, “user_started”, userId);
mSystemServiceManager.startUser(mCurrentUserId);
MountService.onStartUser
mConnector.execute("volume", "user_started", userId);
mHandler.obtainMessage(H_VOLUME_BROADCAST, userVol).sendToTarget();
MountServiceHandler.handleMessage -->case H_VOLUME_BROADCAS
NDC監聽流程:
NDC.istenToSocket
NativeDaemonConnector.handleMessage -->
MountService.onEvent
onEventLocked -->
case VoldResponseCode.VOLUME_CREATED
MountService.onVolumeCreatedLocked
MountServiceHandler.handleMessage -->
case H_VOLUME_MOUNT
mConnector.execute("volume", "mount");
系列文章
Android存儲系統之源碼篇
Android存儲系統之架構篇
來自:http://gityuan.com/2016/07/17/android-io/